Setup
In this notebook we use the following R packages: tidyverse and ggplot2. In addition we use the multiplot functionality provided by Cookbook for R. In Python we use fastai, numpy and pandas. Full code for training and inference available at GitHub. All code for statistical analysis is included in this document. By default all code chunks are hidden. They can easily be expanded by clicking the Code button.
Introduction
The success of deep learning models has typically been measured in terms of their predictive power, but they have lacked a principled way of expressing uncertainty about these predictions.
In my master’s thesis we apply recent insights connecting dropout neural networks to Bayesian approximate variational inference (VI). VI is technique for approximating intractable integrals that arise when modelling the posterior distribution in Bayesian inference and machine learning. Gal et. al. [1, 2, 3] have shown that the posterior distribution of a NN can be approximated by leaving dropout on at test time and sampling multiple predictions. This amounts to drawing Monte Carlo (MC) samples from the posterior (a brief description of this process is listed in section 1.1). The Bayesian framework allows us to obtain principled uncertainty estimates when making predictions in these so called MC dropout NNs.
The results of [1,2,3] seem particularly promising when applied to healthcare. In fact, a recent paper published in Nature [4] applies MC dropout to capture uncertainty estimates when predicting the presence of diabetic retinopathy in patients. This paper demonstrates how uncertainty estimates can provide useful information to the clinician tasked with interpreting the results of medical images. The key idea is that this human-machine interaction will lead to overall better results than either could produce individually.
In this notebook I will first briefly introduce the general idea behind MC dropout. Then we will apply MC dropout in a classification task based on the collection of labelled images in the CIFAR-10 data set.
Background
A neural network is made up of many neurons which are organized in layers. Neurons are often called nodes or units, depending on your choice of machine learning literature. Henceforth we will refer to them as units.
In a typical neural network there are many, many units. As a consequence the number of parameters greatly exceeds the number of data points, dramatically increasing the risk of overfitting (i.e. there are many settings of weights which will cause the network to fit the data almost perfectly).
Dropout is a common stochastic regularisation technique [5] that is used to prevent overfitting in neural networks. The term dropout refers to randomly “dropping out” nonoutput units, temporarily removing all connections to the rest of the network. The main idea is that if the presence of other units is unreliable, each unit is forced to learn how to be useful on its own. At test time all units are activated, and the weights of the network are scaled by the dropout rate in order to match the expected output during training.
Recent work by Gal et. al. [1, 2, 3] casts dropout neural networks as approximate Bayesian inference. Their results show that the predictive posterior distribution of a neural network can be approximated by leaving dropout on at test time.
Consider a classification setting, such as in CIFAR-10. Essentially, what happens when applying MC dropout is the following:
- An image is fed forward through the network \(T\) times (we use \(T=100\) as recommended in [4]). Each time the image is fed through is called a stochastic forward pass.
- For each stochastic forward pass, a slightly different network is making predictions because dropout has randomly switched off units.
- As a result each stochastic forward pass returns 100 slightly different vectors of class predictions.
- To make a prediction we average the 100 vectors. The class corresponding to the largest element in the resulting vector is our final prediction.
- Finally we calculate the standard deviation in class predictions over all forward passes. This is our estimated uncertainty.
Mathematically, model uncertainy is approximated the empirical standard deviation of the predictions for class \(k\), i.e. \[\hat{\sigma}_k = \sqrt{\frac{\sum_{t=1}^T{[p_t(k|X,w) - \hat{\mu}_k}]^2}{T-1}}\] where \[\hat{\mu}_k = \frac{1}{T}\sum_{t=1}^T p_t(k|X,w)\] is the averaged softmax outputs of the predicted class.
Gal et. al. [3] show that the above amounts to drawing Monte Carlo samples from the predictive posterior. Their work demonstrates that applying dropout is effectively the same as defining a Bayesian neural network with a Bernoulli approximating prior over the parameters. Gal has written an excellent blog post that introduces the work and the derivation.
Problem statement
The derivation in Gal et. al. [2] is based on the observation that a shallow neural network with infinitely many weights converges to a Gaussian process (GP) with a specific covariance function. A GP is a non-parametric, probabilistic machine learning method. The idea is that we place a prior distribution over the function space, and by observing new data points we can figure out which function is most likely to have generated the observed data. A GP is fully specified by its mean and covariance functions, and allows us to obtain uncertainty estimates [1]. The extension of this to dropout neural networks is the main result of Gal et. al. [2].
In [1] however, Gal et. al. extend this idea further to include priors over the weights of the convolutional layers in a convolutional neural network (CNN). A CNN is a specialized type of neural network, used primarily and very effectively for image analysis. The authors briefly state that the GP interpretation is lost when the model is extended to CNNs, but that these networks still can be “modelled as Bayesian”.
As far as we are aware, there have been no efforts to determine the correlation between the approximated uncertainty and a network’s ability to predict correctly. Moreover, the work done so far seems to focus on the uncertainty associated with the predicted class. We will examine if there is any additional information to be gained from establishing a connection between the prediction and the runner-up prediction. In other words, our problem statement is:
Are the uncertainty approximations obtained by applying Monte Carlo dropout to convolutional neural networks a reasonable measure of model uncertainty?
Experimental setup
State-of-the-art architectures such as ResNet and DenseNet are very powerful, but they are also complicated and their inner workings are quite convoluted. We are primarily interested in examining the correlation between uncertainty estimation and predictive capabilities. It is arguably better to use a simple network architecture to illustrate the idea of MC dropout. In our approach we use LeNet-5. Click on the tabs for implementation details. Full code is available on GitHub.
Model
LeNet-5 was a pioneering 7-layer convolutional neural network originally developed by Yann LeCunn in 1998 for handwritten digit recognition. It is hopelessly primitive compared to contemporary architectures, but still captures the gist of what a convolutional network is while remaining simple enough to allow us to understand every building block of the network. The following chunk shows the model architecture. Toggle Code button to the right to view.
# Building simple Bayesian CNN with MC dropout and SGD optimizer
def lenet_all(input_shape, nb_classes, p=0.5):
# Building model
model=Sequential([
Conv2D(input_shape=input_shape, filters=192, kernel_size=(5,5)), # 2D convolution layer, e.g. spatial convolution over images.
DropoutMC(p),
MaxPooling2D(strides=2), # Max pooling operation for spatial data
Conv2D(192, kernel_size=(5,5)), # 2D convolution layer, e.g. spatial convolution over images.
DropoutMC(p),
MaxPooling2D(strides=2), # Max pooling operation for spatial data
Flatten(), # Flattens the input
Dense(1000, activation="relu"), # Fully connected layer
DropoutMC(p),
Dense(nb_classes, activation="softmax") # Output layer
])
# Compiling model
model.compile(SGD(momentum=0.9, decay=0.0005), # Optimiser
loss="categorical_crossentropy", # Minimisation objective
metrics=["accuracy"]) # Evaluation metric
return model
MC dropout layer
Our specification of LeNet-5 differs from the orginial in one crucial way: We use Monte Carlo dropout layers. MC dropout is not a feature that is implemented in PyTorch, and we must therefore implemenet one ourselves. Fortunately, this amounts to a simple adjustment of existing code. We modify the Dropout class to take an additional argument called dropoutMC with default value set to True. Toggle Code button to the right to view.
class DropoutMC(Layer):
"""Applies MC Dropout to the input.
Dropout consists in randomly setting
a fraction `rate` of input units to 0 at each update during training time,
which helps prevent overfitting.
MC Dropout consists in sampling from the posterior predictive distribution by activating dropout at test time.
# Arguments
rate: float between 0 and 1. Fraction of the input units to drop.
noise_shape: 1D integer tensor representing the shape of the
binary dropout mask that will be multiplied with the input.
For instance, if your inputs have shape
`(batch_size, timesteps, features)` and
you want the dropout mask to be the same for all timesteps,
you can use `noise_shape=(batch_size, 1, features)`.
seed: A Python integer to use as random seed.
# References
- [Dropout: A Simple Way to Prevent Neural Networks from Overfitting](http://www.cs.toronto.edu/~rsalakhu/papers/srivastava14a.pdf)
"""
@interfaces.legacy_dropout_support
def __init__(self, rate, noise_shape=None, seed=None, **kwargs):
super(DropoutMC, self).__init__(**kwargs)
self.rate = min(1., max(0., rate))
self.noise_shape = noise_shape
self.seed = seed
self.supports_masking = True
def _get_noise_shape(self, inputs):
if self.noise_shape is None:
return self.noise_shape
symbolic_shape = K.shape(inputs)
noise_shape = [symbolic_shape[axis] if shape is None else shape
for axis, shape in enumerate(self.noise_shape)]
return tuple(noise_shape)
def call(self, inputs, training=True):
if 0. < self.rate < 1.:
noise_shape = self._get_noise_shape(inputs)
def dropped_inputs():
return K.dropout(inputs, self.rate, noise_shape,
seed=self.seed)
return K.in_train_phase(dropped_inputs, inputs,
training=training)
return inputs
def get_config(self):
config = {'rate': self.rate,
'noise_shape': self.noise_shape,
'seed': self.seed}
base_config = super(DropoutMC, self).get_config()
return dict(list(base_config.items()) + list(config.items()))
def compute_output_shape(self, input_shape):
return input_shape
Inference
We need to define a function that performs inference over out input. The following chunk contains the relevant code for the sampling procedure described in section 1.1. The function inference stores all the relevant statistics and softmax distributions in a dictionary named output. The results are then turned into a pandas dataframe and some very basic feature engineering is performed. Finally, the data is prepared for statistical analysis in R. Toggle Code button to the right to view.
def batch(img, T=100):
''' Creating mini-batch of T identical images for use in inference.
Arguments:
img, numpy array
T, number of stochastic forward passes (i.e. number of times image needs to be repeated)
'''
img_batch = []
for n in range(0,T):
img_batch.append(img)
return(np.array(img_batch))
def inference(model, X, y, T=100, normalize=False):
''' Function that performs inference by applying MC dropout with T stochastic forward passes on given input.
Arguments:
model, a model object
X, images to be classified
y, corresponding labels,
T, number of stochastic forward passes
'''
# Get images, labels
imgs, labels = X, y.argmax(axis=1)
if normalize:
# Preprocessing input
imgs = imgs.astype("float32")
imgs -= np.mean(X)
imgs /= np.std(X, axis=0)
# Empty dictionary to store all output
output = {}
k=0 # iterator index to keep in dictionary
for (img, label, x) in list(zip(imgs, labels, X)): #added x,X
# Get image batch
img_batch = batch(img, T)
# T predictions on same image
results = model.predict(img_batch, batch_size=T)
# Gathering results
probs = results
probs_mean = np.mean(probs, axis=0)
pred_std = np.std(probs, axis=0)
prediction = probs_mean.argmax()
uncertainty = pred_std[prediction]
correct = 1 if prediction == label else 0
output[k] = {"img": x, "softmax_dist": probs, "probs": probs_mean, "prediction": prediction, "truth": label, "uncertainty": uncertainty, "correct": correct} # added x for img
k+=1
return output
Models
We will examine data gathered from four variants of LeNet-5. All models were trained on CIFAR-10 using the fastai API. CIFAR-10 contains 60.000 labelled 32x32x3 color images belonging to 10 different classes. The input data was split into a training set of 50.000 images and a test set of 10.000 images. The training set was further split into a training set and a validation set. All models have weight_decay = 0.0005 and all learning rates were chosen using lr_find:
model55: Trained for 60 epochs with a learning rate of 0.001 and kernel size = (5,5) and drop_rate = .5. 0.71280 validation loss at end of training with an accuracy of 0.76137 on the validation data. model55 represents the baseline implementation of LeNet-5. It is identical in structure to the one used by Gal et. al. [1].
model52: Trained for 14 epochs with a learning rate of 0.01 for the first 7 and 0.0001 on the remaining. Changed due to rapid overfitting. Model has kernel size = (5,5) and drop_rate = .2. 0.74186 validation loss at end of training with an accuracy of 0.74406 on the validation data.
model35: Trained for 66 epochs with a learning rate of 0.001 and kernel size = (3,3) and drop_rate = .5. 0.75483 validation loss at end of training with an accuracy of 0.74218 on the validation data.
model32: Trained for 52 epochs with a learning rate of 0.001 and kernel size = (3,3) and drop_rate = .2. 0.75449 validation loss at end of training with an accuracy of 0.74090 on the validation data.
Note that model55 has a slightly better baseline performance than the other models.
Data
The data contains the following variables after it has been prepared for analysis in R:
correct (logical): indicator the is TRUE if the predicted class label matches the true class label, else FALSE.
prediction (int): predicted class label.
truth (int): true class label.
uncertainty (dbl): empirical standard deviation of softmax values for predicted class.
prob1 (dbl): argmax of mean softmax output, i.e. mean probability of predicted class.
prob2 (dbl): mean probability of runner-up prediction.
class2 (int): class label of runner-up prediction.
logit_prob1 (dbl): logit transformation of prob1.
diff (dbl): prob1-prob2
diff_sd_ratio (dbl): diff/uncertainty.
All the variables above are pretty standard, with the exception of diff_sd_ratio. Intuitively, if diff is large, the averaged models all agree that class \(k\) is the correct prediction. If diff is small, the models sampled by MC dropout don’t agree on a single class. Thus diff also serves as a proxy for uncertainty. Model uncertainty, however, is approximated by the empirical standard deviation of the predictions for class \(k\). Thus diff_sd_ratio is expressed by \[\tau_{kj} = \frac{\hat{\mu}_k - \hat{\mu}_j}{\hat{\sigma}_k}\] where \(j\) is the runner-up prediction. \(\tau_{jk}\) gives us a ratio of two different measures of uncertainty.
Exploratory analysis
This section will be divided into to parts. First, we will examine the resulting data from model55, which will be regarded as our baseline model. Next, we will analyse the data from models obtained by varying kernel sizes and dropout rates.
Uncertainty analysis of baseline model
Entire data set
# Importing data
data <- as.tibble(read.csv("~/Documents/Masteroppgave/Data/Resultater/results_sgd_df_20180404.csv"))
df <- select(data, -X)
# Summarizing entire data set
summary(df)
correct prediction truth uncertainty prob1 prob2
Min. :0.0000 Min. :0.000 Min. :0.0 Min. :0.0000002 Min. :0.1712 Min. :0.00000
1st Qu.:1.0000 1st Qu.:2.000 1st Qu.:2.0 1st Qu.:0.0361875 1st Qu.:0.6200 1st Qu.:0.01172
Median :1.0000 Median :5.000 Median :4.5 Median :0.1370855 Median :0.8678 Median :0.08034
Mean :0.8381 Mean :4.538 Mean :4.5 Mean :0.1284096 Mean :0.7887 Mean :0.12263
3rd Qu.:1.0000 3rd Qu.:7.000 3rd Qu.:7.0 3rd Qu.:0.2044695 3rd Qu.:0.9819 3rd Qu.:0.21110
Max. :1.0000 Max. :9.000 Max. :9.0 Max. :0.3662840 Max. :1.0000 Max. :0.49695
class2 diff diff_sd_ratio logit_prob1
Min. :0.000 Min. :0.0001831 Min. : 0 Min. :-1.5769
1st Qu.:2.000 1st Qu.:0.3882025 1st Qu.: 2 1st Qu.: 0.4891
Median :3.000 Median :0.7804983 Median : 5 Median : 1.8820
Mean :4.019 Mean :0.6660705 Mean : 1506 Mean : 2.5044
3rd Qu.:6.000 3rd Qu.:0.9697620 3rd Qu.: 27 3rd Qu.: 3.9928
Max. :9.000 Max. :1.0000000 Max. :4859390 Max. :16.3562
NA's :3
For the entire set of classifications, we have the following notable quantities:
The mean uncertainty of the predicted class is 0.1284096 and the median is 0.1370855. The minimum value is 2.0578710^{-7}, the maximum is 0.366284. The interquartile range (IQR) is 0.1682821.
The mean softmax output of the predicted class is 0.7886975 and the median is 0.8678453. The minimum value is 0.1712327, the maximum is 1. The IQR is 0.3618946.
The mean softmax output of the runner-up is 0.122627 and the median is 0.0803399. The minimum value is 3.610851710^{-8}, the maximum is 0.4969542. The IQR is 0.1993783.
The mean difference between the softmax ouputs of the prediction and runner-up is 0.6660705 and the median is 0.7804983. The minimum value is 1.830756710^{-4}, the maximum is 1. The IQR is 0.5815595.
The mean difference to uncertainty ratio, or \(\tau_{jk}\), is 1506.1753085 and the median is 5.4525797. The minimum value is 8.648917410^{-4}, the maximum is 4.859389810^{6}. The IQR is 24.8902696.
Summary statistics
# Aggregating summary statistics by correct/incorrect
agg_df <- df %>%
group_by(correct) %>%
summarise(n=n(),
mean_prob1=mean(prob1),
mean_prob2=mean(prob2),
mean_uncertainty=mean(uncertainty),
median_uncertainty=median(uncertainty),
sd_uncertainty=sd(uncertainty),
mean_diff=mean(diff),
median_diff=median(diff),
sd_diff=sd(diff),
mean_ratio=mean(diff_sd_ratio),
median_ratio=median(diff_sd_ratio),
sd_ratio=sd(diff_sd_ratio))
agg_df
Incorrect classifications
# Summarizing incorrect predictions
incorrect_df <- df %>%
filter(correct==0)
summary(incorrect_df)
correct prediction truth uncertainty prob1 prob2 class2
Min. :0 Min. :0.000 Min. :0.000 Min. :0.0056 Min. :0.1712 Min. :0.0003806 Min. :0.000
1st Qu.:0 1st Qu.:2.000 1st Qu.:2.000 1st Qu.:0.1757 1st Qu.:0.4053 1st Qu.:0.1850289 1st Qu.:2.000
Median :0 Median :4.000 Median :3.000 Median :0.2086 Median :0.5102 Median :0.2487094 Median :4.000
Mean :0 Mean :4.254 Mean :4.022 Mean :0.2077 Mean :0.5321 Mean :0.2532203 Mean :4.053
3rd Qu.:0 3rd Qu.:6.000 3rd Qu.:6.000 3rd Qu.:0.2413 3rd Qu.:0.6354 3rd Qu.:0.3245059 3rd Qu.:5.000
Max. :0 Max. :9.000 Max. :9.000 Max. :0.3663 Max. :0.9991 Max. :0.4925945 Max. :9.000
diff diff_sd_ratio logit_prob1
Min. :0.0001831 Min. : 0.00086 Min. :-1.57692
1st Qu.:0.0943908 1st Qu.: 0.42278 1st Qu.:-0.38356
Median :0.2113830 Median : 0.96571 Median : 0.04078
Mean :0.2788916 Mean : 1.83170 Mean : 0.18488
3rd Qu.:0.4137291 3rd Qu.: 1.96640 3rd Qu.: 0.55535
Max. :0.9986838 Max. :178.33684 Max. : 6.97348
For the entire set of incorrect classifications, we have the following notable quantities:
The mean uncertainty of the predicted class is 0.2076574 and the median is 0.208626. The minimum value is 0.0056, the maximum is 0.366284. The interquartile range (IQR) is 0.065641.
The mean softmax output of the predicted class is 0.5321119 and the median is 0.510194. The minimum value is 0.1712327, the maximum is 0.9990644. The IQR is 0.230109.
The mean softmax output of the runner-up is 0.2532203 and the median is 0.2487094. The minimum value is 3.806021410^{-4}, the maximum is 0.4925945. The IQR is 0.139477.
The mean difference between the softmax ouputs of the prediction and runner-up is 0.2788916 and the median is 0.211383. The minimum value is 1.830756710^{-4}, the maximum is 0.9986838. The IQR is 0.3193383.
The mean difference to uncertainty ratio, or \(\tau_{jk}\), is 1.8317041 and the median is 0.9657115. The minimum value is 8.648917410^{-4}, the maximum is 178.3368412. The IQR is 1.5436196.
Correct classifications
# Summarizing correct predictions
correct_df <- df %>%
filter(correct==1)
summary(correct_df)
correct prediction truth uncertainty prob1 prob2 class2
Min. :1 Min. :0.000 Min. :0.000 Min. :0.0000002 Min. :0.2120 Min. :0.000000 Min. :0.000
1st Qu.:1 1st Qu.:2.000 1st Qu.:2.000 1st Qu.:0.0241575 1st Qu.:0.7323 1st Qu.:0.007342 1st Qu.:2.000
Median :1 Median :5.000 Median :5.000 Median :0.1052500 Median :0.9171 Median :0.049465 Median :3.000
Mean :1 Mean :4.592 Mean :4.592 Mean :0.1131009 Mean :0.8383 Mean :0.097400 Mean :4.012
3rd Qu.:1 3rd Qu.:7.000 3rd Qu.:7.000 3rd Qu.:0.1883090 3rd Qu.:0.9888 3rd Qu.:0.160019 3rd Qu.:6.000
Max. :1 Max. :9.000 Max. :9.000 Max. :0.3562930 Max. :1.0000 Max. :0.496954 Max. :9.000
diff diff_sd_ratio logit_prob1
Min. :0.0003563 Min. : 0 Min. :-1.313
1st Qu.:0.5665170 1st Qu.: 3 1st Qu.: 1.005
Median :0.8653638 Median : 8 Median : 2.402
Mean :0.7408638 Mean : 1797 Mean : 2.953
3rd Qu.:0.9809582 3rd Qu.: 40 3rd Qu.: 4.475
Max. :1.0000000 Max. :4859390 Max. :16.356
NA's :3
For the entire set of correct classifications, we have the following notable quantities:
The mean uncertainty of the predicted class is 0.1131009 and the median is 0.10525. The minimum value is 2.0578710^{-7}, the maximum is 0.356293. The interquartile range (IQR) is 0.1641515.
The mean softmax output of the predicted class is 0.8382634 and the median is 0.917101. The minimum value is 0.2119797, the maximum is 1. The IQR is 0.2565594.
The mean softmax output of the runner-up is 0.0973997 and the median is 0.0494654. The minimum value is 3.610851710^{-8}, the maximum is 0.4969542. The IQR is 0.1526768.
The mean difference between the softmax ouputs of the prediction and runner-up is 0.7408638 and the median is 0.8653638. The minimum value is 3.562569610^{-4}, the maximum is 1. The IQR is 0.4144412.
The mean difference to uncertainty ratio, or \(\tau_{jk}\), is 1796.7769426 and the median is 8.1020167. The minimum value is 0.0019334, the maximum is 4.859389810^{6}. The IQR is 37.632868.
Distribution of uncertainty
In the following we will visualize the relationships between our variabels. We start by examining the empirical distribution of the uncertainty estimates \(\hat{\sigma}_k\).
Full distribution
The distribution appears to be bimodal, with peaks close to 0 and 0.2:
# Distribution of estimated uncertainty
p1 <- df %>%
ggplot(aes(x=uncertainty)) +
geom_histogram(col="grey", bins = 50, alpha=.5) +
ggtitle("Distribution of estimated uncertainty")
p1

Uncertainty by prediction
By grouping the uncertainty estimates by correct (i.e. if the label was correctly predicted or not), we can find out how the predictions contribute to the uncertainty distribution.
The blue line corresponds to the correct predictions, the red line corresponds to incorrect predictions. We see that the incorrect predictions are centered around a higher associated uncertainty, whereas far more of the correctly predicted classes are concentrated around a low uncertainty value. The incorrect classifications greatly contribute to the bimodality, but it is also present in the distribution of uncertainty for the correct classifications.
#Distribution of estimated uncertainty by prediction
p2 <- df %>%
ggplot(aes(x=uncertainty, col=factor(correct))) +
geom_freqpoly(alpha=.7) +
ggtitle("Distribution of estimated uncertainty by classification") +
scale_color_discrete(name="Prediction",
breaks=c("0", "1"),
labels=c("0: Incorrect", "1: Correct"))
p2

Kernel density estimates
The following kernel density estimate plot (using a Gaussian kernel) gives us an idea of how the distributions compare to eachother:
# KDE by correct prediction
p3 <- df %>%
ggplot(aes(x=uncertainty)) +
geom_density(data=incorrect_df, fill="red", alpha=I(.2)) +
geom_density(data=correct_df, fill="turquoise", alpha=I(.2)) +
ggtitle("Kernel density estimates of uncertainty distribution")
p3

Boxplots
The boxplot gives us yet another way to visualize the difference between uncertainty distributions by predictive ability. It is interesting to note the amount of outliers for the incorrect predictions, indicating som negative skew. It is wrong, but uncertainty is low.
# Boxplot of uncertainties for correct vs. incorrect
p4 <- df %>%
ggplot(aes(x=factor(correct), y=uncertainty)) +
geom_boxplot(aes(fill=factor(correct)), alpha=.7) +
labs(x="correct") +
ggtitle("Boxplot of uncertainty distribution by correct/incorrect") +
scale_fill_discrete(name="Prediction",
breaks=c("0", "1"),
labels=c("0: Incorrect", "1: Correct"))
p4

Relationship to other variables
Softmax of predicted class
We may also be interested in the relationship between uncertainty and other variables. First, we plot uncertainty against prob1 (the prediction’s softmax output). The softmax outputs in the above plot are colour graded. Outputs close to 1 are red, outputs close to 0 are blue. The plot shows a clear parabolic shape:
# Plotting softmax output of prediction against estimated uncertainty
p5 <- df %>%
ggplot(aes(x=prob1, y=uncertainty)) +
geom_point(alpha=.2) +
ggtitle("Uncertainty vs. softmax output of prediction for all observations") +
labs(x="softmax output of predicted class") +
scale_color_distiller(name="Softmax output",
palette = "Spectral")
p5

By classification
Red points indicate incorrect classifications, blue points indicate correct classifications.
p6 <- df %>%
mutate(correct=as.logical(correct)) %>%
ggplot(aes(x=prob1, y=uncertainty)) +
geom_point(aes(fill=correct, col=correct), shape=21, alpha=.5) +
ggtitle("Uncertainty vs. softmax output of prediction for all observations") +
labs(x="softmax output of predicted class")
p6

Predictions and runners-up
We can obtain more information by colouring the points by the value of the runner-up predictions. This plot is particularly interesting. The softmax outputs of the runner-up classes \(\hat{\mu}_j\) in the above plot are colour graded. \(\hat{\mu}_j \approx 0.5\) are red, \(\hat{\mu}_j \approx 0\) are blue. The round point is the pair mean values of (prob1, prob2) for the incorrect predictions. The triangular point is the pair mean values for the correct predictions.
We see a clear concentration of red points in the area where the probability of the predictied class \(\hat{\mu}_k \approx 0.5\). If the softmax predictions of both the predicted class and the runner-up are close 0.5, then we have a situation analogous to maximum entropy. This points coincide with the largest approximated uncertainty values.
# Plotting softmax output of prediction against estimated uncertainty, coloured by softmax output of runner-up
p7 <- df %>%
ggplot(aes(x=prob1, y=uncertainty)) +
geom_point(aes(col=prob2), shape=21, alpha=.7) +
geom_point(data=agg_df, aes(x=mean_prob1, y=mean_prob2, shape=as.logical(correct)), size=2.5) +
#geom_point(data=agg_df, aes(y=uncertainty, x=mean_prob1, shape=as.logical(correct)), size=2) +
ggtitle("Uncertainty vs. softmax output of prediction for all observations") +
labs(x="softmax output of predicted class", shape="correct") +
scale_color_distiller(name="runner up",
palette = "Spectral")
p7

Softmax of prediction by classification
In the following plot we split the observations by incorrect/correct predictions, and plot the values of uncertainty against the softmax output of the predicted class:
# Plotting uncertainty vs. softmax output of prediction
p8a <- df %>%
ggplot(aes(x=prob1, y=uncertainty)) +
geom_point(aes(col=prob2), shape=21, alpha=.7) +
ggtitle("Uncertainty vs. softmax output of prediction by incorrect/correct") +
labs(x="softmax output of prediction") +
facet_grid(.~factor(correct)) +
scale_color_distiller(name="Runner up",
palette = "Spectral")
p8a

Softmax of prediction by classification with contours
The black contour lines indicate where most of the points are concentrated. The plot on the left is for incorrect predictions. The right hand plot represents the correct predictions. For the correct predictions, it seems as if far more of the points are concentrated around high predicted output/low runner-up output/low uncertainty. This is not surprising, considering 75% of the correct predictions have a softmax value of approximately 0.7 or above. For the incorrect classifications, most of the points are concentrated around the area of maximum entropy. This indicates that the approximated uncertainty estimates indeed contain valuable information in the incorrect cases.
# Plotting uncertainty vs. softmax output of prediction
p8 <- df %>%
ggplot(aes(x=prob1, y=uncertainty)) +
geom_point(aes(col=prob2), shape=21, alpha=.7) +
geom_density_2d(col="black", alpha=.3) +
ggtitle("Uncertainty vs. softmax output of prediction by incorrect/correct") +
labs(x="softmax output of prediction") +
facet_grid(.~factor(correct)) +
scale_color_distiller(name="Runner up",
palette = "Spectral")
p8

Runner-up predictions
The following plot shows the relationship between uncertainty estimates and the softmax output of the runner-up prediction. Unsurprisingly, model uncertainty increases as the softmax output of the runner-up increases. We have plotted a LOESS estimate of the mean uncertainty as a function of the runner-up output to make this clearer:
# Plotting softmax output of runner-up against estimated uncertainty
p10 <- df %>%
ggplot(aes(x=prob2, y=uncertainty)) +
geom_point(alpha=.2) +
geom_smooth(method="loess") +
labs(x="softmax output of runner-up") +
ggtitle("Uncertainty vs. softmax output of runner-up")
p10

Images associated with high uncertainty
The following plots were generated using Python (code available on GitHub). On the left hand side we see the unnormalized image with the corresponding ground truth label. The plot in the middle shows the softmax output of the predicted class for each of the \(T=100\) stochastic forward passes. \(\mu_k\) is given by the solid red line, \(\mu_j\) is given by the dashed brown line. The plot title shows both the predicted class and the runner-up class. To the right is a kernel density estimate (using a Gaussian kernel) of the \(T=100\) softmax outputs for the predicted class. The plots show the top 5 most uncertain classifications in the entire data set.
NOTE: Add new images.
Uncertainty-prediction correlation
As mentioned in section 2.2, the connection established between dropout neural networks and GPs are lost when applied to convolutional neural networks. Performing a logistic regression gives us a simple way of testing if the approximated uncertainty is a significant predictor of the model’s ability to predict correctly.
The coefficient associated with uncertainty is highly significant, indicating that the estimates gathered from performing MC dropout are indeed a useful quantification of predictive uncertainty. Given a unit increase in uncertainty, we expect the probability of predicting correctly to decrease.
# Fitting logistic regression model to check significance of uncertainty
model_sd <- glm(factor(correct)~uncertainty, data=df, family = binomial(link="logit"))
summary(model_sd)
Call:
glm(formula = factor(correct) ~ uncertainty, family = binomial(link = "logit"),
data = df)
Deviance Residuals:
Min 1Q Median 3Q Max
-2.8062 0.1968 0.3105 0.6347 1.6688
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 3.99779 0.08754 45.67 <2e-16 ***
uncertainty -14.32714 0.42711 -33.55 <2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 8856.1 on 9999 degrees of freedom
Residual deviance: 7234.4 on 9998 degrees of freedom
AIC: 7238.4
Number of Fisher Scoring iterations: 5
Referral criteria
The question is then: How do we determine a reasonable uncertainty value for referral to a human expert? As we saw in section 3.1.2, the mean uncertainty values of the incorrect predictions higher than for the correct predictions (0.2076574 vs. 0.1131009, respectively).
Naively, we could set the threshold for referral to the mean uncertainty of the incorrectly classified images:
# Counting number of correct/incorrect by uncertainty >= .207
referral_mean <- df %>%
filter(uncertainty>=.207) %>%
count(correct)
referral_mean
The problem here is apparent: Many of the correctly classified images are associated with a relatively high level of uncertainty (in our case, this is due to the bimodality of the uncertainty distribution in section 3.2.1). The proportion of incorrectly classified images in the referred sample is 0.346845.
The result of our logistic regression suggests that uncertainty is a significant predictor of a model’s ability to predict correctly. However, although uncertainty seems to contain valueable information, the relative uncertainties of the correct/incorrect observations (in this case) may be too small to differentiate which images should be referred to an expert. We may need to amplify the quantification of uncertainty in some way.
Usefulness of runner-up predictions
For the most uncertain incorrectly classified images in section 3.4.1 the runner-up suggestions are in fact the ground truth labels 80% of the time. This may be due to chance, but still begs the question: Is there any information to be obtained from the runner-up predictions? What would happen to our overall accuracy if we used the runner-up predictions for all incorrect classifications?
# Counting classification accuracy if runner-up is equal to ground truth
class2_df <- df %>%
mutate(correct=replace(correct, correct==0 & class2==truth, 1)) %>%
count(correct)
class2_df
Accuracy would rise to 0.9393. This indicates that there may be some valuable information to be gathered from the runner-up predictions.
A revised quantification of uncertainty
One possible approach is to use the value of \(\tau_{jk}\) introduced in section 2.5. Recall that \(\tau_{jk}\) incorporates information about the runner-up prediction into our quantification of uncertainty. Consider the following cases:
\(\tau_{kj} \approx 1\) means that diff and uncertainty are relatively similar. This happens if a) the models have failed to reach a consensus (diff is small) but model uncertainty is low, or b) the models have reached a consensus (diff is large) but model uncertainty is high. Let’s call these referral predictions.
\(\tau_{kj} \to 0\) means that uncertainty is much larger than diff. These should represent uncertain predictions.
\(\tau_{kj} \to \infty\) means that uncertainty is much smaller than diff. These should represent non-referral predictions.
As we saw in table 3.1.2, the mean \(\tau_{jk}\) for the incorrectly classified images is 1.8317041 and 1796.7769426 for the correctly classified images. The respective medians are 0.9657115 and 8.1020167.
As a first step, setting the referral threshold to \(\tau_{jk} \leq 1.83\) (the mean uncertainty of the incorrect classifications) gives:
# Counting number of correct/incorrect by tau <= 2.8
referral_meantau <- df %>%
filter(diff_sd_ratio <= 1.83) %>%
count(correct)
referral_meantau
We run into the same problem as when we used the mean uncertainty: Many of the correctly predicted images would be referred. It is interesting to note, however, that we get more referrals of incorrectly classified images. In this case, the proportion of incorrectly classified images in the referred sample is 0.4581071.
Upon inspection, the boxplot of the log-transformed \(\tau_{jk}\) clearly shows the presence of extreme values (outliers marked in red) in both tails. For incorrect predictions, the distribution seems to be negatively skewed. The distribution of the correct predictions appear to be positively skewed.
pt <- df %>%
ggplot(aes(x=factor(correct), y=log(diff_sd_ratio))) +
geom_boxplot(aes(fill=factor(correct)), outlier.colour = "red", outlier.alpha = .15) +
xlab("correct") +
ylab("log-transformed tau") +
ggtitle("Presence of extreme uncertainty values") +
labs(fill="correct")
pt

Perhaps a more robust measure of the central tendency of \(\tau_{jk}\) is the median. Setting the referral rate of \(\tau_{jk} \leq 1.2\) gives the following:
# Counting number of correct/incorrect by tau <= 1
referral_medtau <- df %>%
filter(diff_sd_ratio <= .965) %>%
count(correct)
referral_medtau
This is a clear improvement over our the mean approach, in terms of the amount of referred images. 373 fewer incorrect images are referred compared to 673 fewer correct images. In this case, the proportion of incorrectly classified images in the referred sample is 0.5274151.
When compared to using the mean of \(\hat{\sigma}_k\) as a cut-off for referral, the median of \(\tau_{jk}\) results in almost 20% more referrals of incorrect image relative to the sample size. This could indicate that \(\tau_{jk}\) is a more sensitive uncertainty quantification for practical applications.
Comparison of tau and uncertainty
We can examine this further by plotting the number of correct/incorrect referrals against different values for \(\tau_{jk}\) and \(\sigma_k\).
Data preparation
First we need to prepare the data for plotting. In the following chunk we count the number of referrals of correctly/incorrectly classified images by increasing the values of \(\tau_{jk}\) and \(\sigma_k\):
For \(\tau_{jk}\), we set the initial value \(\tau_1 = 0.05\) and increment by 0.01 until we reach a maximum value of 10. This cut-off value was chosen because it gives us roughly the same proportion of correct vs. incorrect classifications as our threshold for \(\sigma_k\). For every \(\tau_{jk} \leq \tau_i\) we count the number correct and incorrect classifications of the referred images. Finally, we calculate the relative proportions of correctly/incorrectly classified images per value.
For \(\sigma_k\), we set the initial value to \(\sigma_1=0.01\) and increment by 0.01 until we reach a maximum value of .35. This cut-off value was chosen because it gives us roughly the same proportion of correct vs. incorrect classifications as our threshold for \(\tau_{jk}\). For every \(\sigma_{k} \geq \sigma_i\) (i.e. all images that have higher uncertainty than this value) we count the number correct and incorrect classifications of the referred images.
# Preparing data
count_df <- df %>%
rename(tau=diff_sd_ratio) %>%
select(correct, uncertainty, tau)
# Gathering number of referrals by tau
roc_tau <- data.frame(n_false=numeric(), n_true=numeric(), tau_value=numeric())
idx=1
tau_seq <- seq(0.05, 10, by=.1)
for(value in tau_seq){
get_results <- count_df %>%
filter(tau<=value) %>%
count(correct)
roc_tau[idx,] <- c(get_results$n, value)
idx <- idx + 1
}
plot_tau <- roc_tau %>%
gather(status, n, -tau_value) %>%
group_by(tau_value) %>%
mutate(p=n/sum(n)) %>%
ungroup()
## Gathering number of referrals per by uncertainty
roc_unc <- data.frame(n_false=numeric(), n_true=numeric(), uncertainty_value=numeric())
idx=1
uncertainty_seq <- seq(0.01, .35, by=.01)
for(value in uncertainty_seq){
get_results <- count_df %>%
filter(uncertainty>=value) %>%
count(correct)
roc_unc[idx,] <- c(get_results$n, value)
idx <- idx + 1
}
plot_unc <- roc_unc %>%
gather(status, n, -uncertainty_value) %>%
group_by(uncertainty_value) %>%
mutate(p=n/sum(n)) %>%
ungroup()
Plotting
The first row of the plot contains the referrals for varying values of \(\sigma_k\). The second row shows referral counts for varying values of \(\tau_{jk}\). The first column show the absolute counts (note the different x-axes) and the second column shows the relative referral rates. The dashed horisontal lines represent the mean of \(\sigma_k\) and the median of \(\tau_{jk}\), respectively, for the incorrectly classified images.
# Absolute counts
p1a <- ggplot(plot_unc, aes(x=uncertainty_value, y=n, group=status)) +
geom_line(aes(col=status)) +
geom_vline(xintercept=agg_df$mean_uncertainty[1], lwd=.3, lty="dashed") +
labs(x="uncertainty", y="number of referrals") +
ggtitle("Absolute referral counts") +
theme(legend.position = "none")
p2a <- ggplot(plot_tau, aes(x=tau_value, y=n, group=status)) +
geom_line(aes(col=status)) +
geom_vline(xintercept=agg_df$median_ratio[1], lwd=.3, lty="dashed") +
labs(x="tau", y="number of referrals") +
ggtitle("Absolute referral counts")+
theme(legend.position = "none")
# Relative counts
p1b <- ggplot(plot_unc, aes(x=uncertainty_value, y=p, group=status)) +
geom_line(aes(col=status)) +
labs(x="uncertainty", y="proportion of referrals") +
geom_vline(xintercept=agg_df$mean_uncertainty[1], lwd=.3, lty="dashed") +
ggtitle("Relative referral rates")+
theme(legend.position = "none")
p2b <- ggplot(plot_tau, aes(x=tau_value, y=p, group=status)) +
geom_line(aes(col=status)) +
labs(x="tau", y="proportion of referrals") +
geom_vline(xintercept=agg_df$median_ratio[1], lwd=.3, lty="dashed") +
ggtitle("Relative referral rates")+
theme(legend.position = "none")
layout <- matrix(c(1,2,3,4), nrow = 2, ncol = 2, byrow=TRUE)
multiplot(p1a, p1b, p2a, p2b, layout = layout)

Tau vs. uncertainty
Plotting the absolute referral counts for varying values of \(\tau_{jk}\) (blue line) and \(\sigma_k\) (red line) in the same plot tells as that \(\tau_{jk}\) gives us more “bang for the buck”, so to speak.
p3a <- ggplot(roc_tau, aes(x=n_true, y=n_false)) +
geom_line(col="blue") +
geom_line(data=roc_unc, aes(x=n_true, y=n_false), col="red") +
ylim(c(0,2000)) +
xlim(c(0,8000))
p3a

Digit classification on MNIST
Draft: Checking to see if \(\tau_{jk}\) is useful on a completely different data set (MNIST).
# Importing MNIST data
data_mnist <- as.tibble(read.csv("~/Documents/Masteroppgave/Data/Resultater/mnist_df.csv"))
df_mnist <- select(data_mnist, -X)
# Summarizing entire data set
summary(df_mnist)
correct prediction truth uncertainty prob1 prob2
Min. :0.0000 Min. :0.000 Min. :0.000 Min. :0.0000011 Min. :0.2871 Min. :0.0000002
1st Qu.:1.0000 1st Qu.:2.000 1st Qu.:2.000 1st Qu.:0.0004957 1st Qu.:0.9928 1st Qu.:0.0001036
Median :1.0000 Median :4.000 Median :4.000 Median :0.0024089 Median :0.9991 Median :0.0005784
Mean :0.9864 Mean :4.433 Mean :4.443 Mean :0.0265448 Mean :0.9747 Mean :0.0186610
3rd Qu.:1.0000 3rd Qu.:7.000 3rd Qu.:7.000 3rd Qu.:0.0166062 3rd Qu.:0.9998 3rd Qu.:0.0047357
Max. :1.0000 Max. :9.000 Max. :9.000 Max. :0.3386140 Max. :1.0000 Max. :0.4948297
class2 diff diff_sd_ratio logit_prob1
Min. :0.000 Min. :0.0002854 Min. : 0.0 Min. :-0.9096
1st Qu.:2.000 1st Qu.:0.9881017 1st Qu.: 59.2 1st Qu.: 4.9280
Median :5.000 Median :0.9984862 Median : 414.4 Median : 6.9962
Mean :5.119 Mean :0.9560544 Mean : 4602.0 Mean : 6.7159
3rd Qu.:7.000 3rd Qu.:0.9997167 3rd Qu.: 2016.7 3rd Qu.: 8.6554
Max. :9.000 Max. :0.9999995 Max. :899714.7 Max. :15.4349
# Aggregating summary statistics by correct/incorrect
agg_df_mnist <- df_mnist %>%
group_by(correct) %>%
summarise(n=n(),
mean_prob1=mean(prob1),
mean_prob2=mean(prob2),
mean_uncertainty=mean(uncertainty),
median_uncertainty=median(uncertainty),
sd_uncertainty=sd(uncertainty),
mean_diff=mean(diff),
median_diff=median(diff),
sd_diff=sd(diff),
mean_tau=mean(diff_sd_ratio),
median_tau=median(diff_sd_ratio),
sd_ratio=sd(diff_sd_ratio))
agg_df_mnist
# Finding cases where runner up equals true class
medtau <- df_mnist %>%
filter(class2 == truth) %>%
summarise(n = n(),
mean_uncertainty = mean(uncertainty),
median_uncertainty = median(uncertainty),
mean_tau = mean(diff_sd_ratio),
median_tau = median(diff_sd_ratio))
medtau
# Setting referral rate to mean uncertainty of incorrect predictions
referral_mnist_sigma <- df_mnist %>%
filter(uncertainty >= .22) %>%
count(correct)
referral_mnist_sigma
In this case, the proportion of incorrectly classified images in the referred sample is 0.2666667.
# Setting referral rate to mean uncertainty of incorrect predictions
referral_mnist_tau <- df_mnist %>%
filter(diff_sd_ratio <= 1.36) %>%
count(correct)
referral_mnist_tau
In this case, the proportion of incorrectly classified images in the referred sample is 0.3941176. Again, preliminary results suggest that runner-up predictions can be leveraged to produce sensible cut-off values for referral.
LS0tCnRpdGxlOiAiQW5hbHlzaXMgb2YgYXBwcm94aW1hdGVkIHVuY2VydGFpbnR5IGluIGRlZXAgbGVhcm5pbmciCmF1dGhvcjogIlNlYW4gTWVsaW5nIE11cnJheSIKb3V0cHV0OgogIGh0bWxfZG9jdW1lbnQ6CiAgICBkZl9wcmludDogcGFnZWQKICAgIHRvYzogeWVzCiAgICBjb2RlX2ZvbGRpbmc6IGhpZGUKICBodG1sX25vdGVib29rOgogICAgbnVtYmVyX3NlY3Rpb25zOiB5ZXMKICAgIHRvYzogeWVzCiAgICBjb2RlX2ZvbGRpbmc6IGhpZGUKLS0tCgoqKioKIyBTZXR1cApJbiB0aGlzIG5vdGVib29rIHdlIHVzZSB0aGUgZm9sbG93aW5nIFIgcGFja2FnZXM6IGB0aWR5dmVyc2VgIGFuZCBgZ2dwbG90MmAuIEluIGFkZGl0aW9uIHdlIHVzZSB0aGUgYG11bHRpcGxvdGAgZnVuY3Rpb25hbGl0eSBwcm92aWRlZCBieSBbQ29va2Jvb2sgZm9yIFJdKGh0dHA6Ly93d3cuY29va2Jvb2stci5jb20vR3JhcGhzL011bHRpcGxlX2dyYXBoc19vbl9vbmVfcGFnZV8oZ2dwbG90MikvKS4gSW4gUHl0aG9uIHdlIHVzZSBgZmFzdGFpYCwgYG51bXB5YCBhbmQgYHBhbmRhc2AuIEZ1bGwgY29kZSBmb3IgdHJhaW5pbmcgYW5kIGluZmVyZW5jZSBbYXZhaWxhYmxlIGF0IEdpdEh1Yl0oaHR0cHM6Ly9naXRodWIuY29tL3NtdTA5NS9zbXUwOTUuZ2l0aHViLmlvL2Jsb2IvbWFzdGVyL2RhdDI1OS5pcHluYikuIEFsbCBjb2RlIGZvciBzdGF0aXN0aWNhbCBhbmFseXNpcyBpcyBpbmNsdWRlZCBpbiB0aGlzIGRvY3VtZW50LiBCeSBkZWZhdWx0IGFsbCBjb2RlIGNodW5rcyBhcmUgaGlkZGVuLiBUaGV5IGNhbiBlYXNpbHkgYmUgZXhwYW5kZWQgYnkgY2xpY2tpbmcgdGhlIGBDb2RlYCBidXR0b24uCgpgYGB7ciBlY2hvPUZBTFNFLCByZXN1bHRzPSdoaWRlJ30KIyBMb2FkaW5nIHVzZWZ1bCBwYWNrYWdlcwpsaWJyYXJ5KHRpZHl2ZXJzZSkKbGlicmFyeShnZ3Bsb3QyKQpsaWJyYXJ5KEhtaXNjKQpsaWJyYXJ5KGtuaXRyKQojbGlicmFyeShrYWJsZUV4dHJhKQoKIyBNdWx0aXBsZSBwbG90IGZ1bmN0aW9uCiMKIyBnZ3Bsb3Qgb2JqZWN0cyBjYW4gYmUgcGFzc2VkIGluIC4uLiwgb3IgdG8gcGxvdGxpc3QgKGFzIGEgbGlzdCBvZiBnZ3Bsb3Qgb2JqZWN0cykKIyAtIGNvbHM6ICAgTnVtYmVyIG9mIGNvbHVtbnMgaW4gbGF5b3V0CiMgLSBsYXlvdXQ6IEEgbWF0cml4IHNwZWNpZnlpbmcgdGhlIGxheW91dC4gSWYgcHJlc2VudCwgJ2NvbHMnIGlzIGlnbm9yZWQuCiMKIyBJZiB0aGUgbGF5b3V0IGlzIHNvbWV0aGluZyBsaWtlIG1hdHJpeChjKDEsMiwzLDMpLCBucm93PTIsIGJ5cm93PVRSVUUpLAojIHRoZW4gcGxvdCAxIHdpbGwgZ28gaW4gdGhlIHVwcGVyIGxlZnQsIDIgd2lsbCBnbyBpbiB0aGUgdXBwZXIgcmlnaHQsIGFuZAojIDMgd2lsbCBnbyBhbGwgdGhlIHdheSBhY3Jvc3MgdGhlIGJvdHRvbS4KIwptdWx0aXBsb3QgPC0gZnVuY3Rpb24oLi4uLCBwbG90bGlzdD1OVUxMLCBmaWxlLCBjb2xzPTEsIGxheW91dD1OVUxMKSB7CiAgbGlicmFyeShncmlkKQoKICAjIE1ha2UgYSBsaXN0IGZyb20gdGhlIC4uLiBhcmd1bWVudHMgYW5kIHBsb3RsaXN0CiAgcGxvdHMgPC0gYyhsaXN0KC4uLiksIHBsb3RsaXN0KQoKICBudW1QbG90cyA9IGxlbmd0aChwbG90cykKCiAgIyBJZiBsYXlvdXQgaXMgTlVMTCwgdGhlbiB1c2UgJ2NvbHMnIHRvIGRldGVybWluZSBsYXlvdXQKICBpZiAoaXMubnVsbChsYXlvdXQpKSB7CiAgICAjIE1ha2UgdGhlIHBhbmVsCiAgICAjIG5jb2w6IE51bWJlciBvZiBjb2x1bW5zIG9mIHBsb3RzCiAgICAjIG5yb3c6IE51bWJlciBvZiByb3dzIG5lZWRlZCwgY2FsY3VsYXRlZCBmcm9tICMgb2YgY29scwogICAgbGF5b3V0IDwtIG1hdHJpeChzZXEoMSwgY29scyAqIGNlaWxpbmcobnVtUGxvdHMvY29scykpLAogICAgICAgICAgICAgICAgICAgIG5jb2wgPSBjb2xzLCBucm93ID0gY2VpbGluZyhudW1QbG90cy9jb2xzKSkKICB9CgogaWYgKG51bVBsb3RzPT0xKSB7CiAgICBwcmludChwbG90c1tbMV1dKQoKICB9IGVsc2UgewogICAgIyBTZXQgdXAgdGhlIHBhZ2UKICAgIGdyaWQubmV3cGFnZSgpCiAgICBwdXNoVmlld3BvcnQodmlld3BvcnQobGF5b3V0ID0gZ3JpZC5sYXlvdXQobnJvdyhsYXlvdXQpLCBuY29sKGxheW91dCkpKSkKCiAgICAjIE1ha2UgZWFjaCBwbG90LCBpbiB0aGUgY29ycmVjdCBsb2NhdGlvbgogICAgZm9yIChpIGluIDE6bnVtUGxvdHMpIHsKICAgICAgIyBHZXQgdGhlIGksaiBtYXRyaXggcG9zaXRpb25zIG9mIHRoZSByZWdpb25zIHRoYXQgY29udGFpbiB0aGlzIHN1YnBsb3QKICAgICAgbWF0Y2hpZHggPC0gYXMuZGF0YS5mcmFtZSh3aGljaChsYXlvdXQgPT0gaSwgYXJyLmluZCA9IFRSVUUpKQoKICAgICAgcHJpbnQocGxvdHNbW2ldXSwgdnAgPSB2aWV3cG9ydChsYXlvdXQucG9zLnJvdyA9IG1hdGNoaWR4JHJvdywKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBsYXlvdXQucG9zLmNvbCA9IG1hdGNoaWR4JGNvbCkpCiAgICB9CiAgfQp9CgojIERhdGEgY2xlYW5pbmcgZnVuY3Rpb24KcHJvY2VzcyA8LSBmdW5jdGlvbihST09ULCBmaWxlbmFtZSwgbmFtZSwga2VybmVsX3N6LCBkcm9wX3JhdGUpewogIFJPT1QgPC0gIn4vRG9jdW1lbnRzL01hc3Rlcm9wcGdhdmUvRGF0YS9SZXN1bHRhdGVyLyIKICBQQVRIIDwtIHBhc3RlKFJPT1QsIGZpbGVuYW1lLCBzZXA9IiIpCiAgZGF0IDwtIGFzLnRpYmJsZShyZWFkLmNzdihhcy5jaGFyYWN0ZXIoUEFUSCkpKQogICNkYXQgPC0gc2VsZWN0KGRhdCwgLVgpCiAgZGF0IDwtIGRhdCAlPiUgCiAgICBtdXRhdGUobW9kZWw9bmFtZSwKICAgICAgICAgICBrZXJuZWw9a2VybmVsX3N6LAogICAgICAgICAgIGRyb3BvdXQ9ZHJvcF9yYXRlKQogIHJldHVybihkYXQpCn0KYGBgCgojIEludHJvZHVjdGlvbgoKVGhlIHN1Y2Nlc3Mgb2YgZGVlcCBsZWFybmluZyBtb2RlbHMgaGFzIHR5cGljYWxseSBiZWVuIG1lYXN1cmVkIGluIHRlcm1zIG9mIHRoZWlyIHByZWRpY3RpdmUgcG93ZXIsIGJ1dCB0aGV5IGhhdmUgbGFja2VkIGEgcHJpbmNpcGxlZCB3YXkgb2YgZXhwcmVzc2luZyB1bmNlcnRhaW50eSBhYm91dCB0aGVzZSBwcmVkaWN0aW9ucy4KCkluIG15IG1hc3RlcidzIHRoZXNpcyB3ZSBhcHBseSByZWNlbnQgaW5zaWdodHMgY29ubmVjdGluZyBkcm9wb3V0IG5ldXJhbCBuZXR3b3JrcyB0byBbQmF5ZXNpYW4gYXBwcm94aW1hdGUgdmFyaWF0aW9uYWwgaW5mZXJlbmNlIChWSSldKGh0dHBzOi8vYXJ4aXYub3JnL2Ficy8xNjAxLjAwNjcwKS4gVkkgaXMgdGVjaG5pcXVlIGZvciBhcHByb3hpbWF0aW5nIGludHJhY3RhYmxlIGludGVncmFscyB0aGF0IGFyaXNlIHdoZW4gbW9kZWxsaW5nIHRoZSBwb3N0ZXJpb3IgZGlzdHJpYnV0aW9uIGluIEJheWVzaWFuIGluZmVyZW5jZSBhbmQgbWFjaGluZSBsZWFybmluZy4gR2FsIGV0LiBhbC4gWzEsIDIsIDNdIGhhdmUgc2hvd24gdGhhdCB0aGUgcG9zdGVyaW9yIGRpc3RyaWJ1dGlvbiBvZiBhIE5OIGNhbiBiZSBhcHByb3hpbWF0ZWQgYnkgbGVhdmluZyBkcm9wb3V0IG9uIGF0IHRlc3QgdGltZSBhbmQgc2FtcGxpbmcgbXVsdGlwbGUgcHJlZGljdGlvbnMuIFRoaXMgYW1vdW50cyB0byBkcmF3aW5nIE1vbnRlIENhcmxvIChNQykgc2FtcGxlcyBmcm9tIHRoZSBwb3N0ZXJpb3IgKGEgYnJpZWYgZGVzY3JpcHRpb24gb2YgdGhpcyBwcm9jZXNzIGlzIGxpc3RlZCBpbiBzZWN0aW9uIDEuMSkuIFRoZSBCYXllc2lhbiBmcmFtZXdvcmsgYWxsb3dzIHVzIHRvIG9idGFpbiBwcmluY2lwbGVkIHVuY2VydGFpbnR5IGVzdGltYXRlcyB3aGVuIG1ha2luZyBwcmVkaWN0aW9ucyBpbiB0aGVzZSBzbyBjYWxsZWQgKk1DIGRyb3BvdXQqIE5Ocy4gCgpUaGUgcmVzdWx0cyBvZiBbMSwyLDNdIHNlZW0gcGFydGljdWxhcmx5IHByb21pc2luZyB3aGVuIGFwcGxpZWQgdG8gaGVhbHRoY2FyZS4gSW4gZmFjdCwgYSByZWNlbnQgcGFwZXIgcHVibGlzaGVkIGluIE5hdHVyZSBbNF0gYXBwbGllcyBNQyBkcm9wb3V0IHRvIGNhcHR1cmUgdW5jZXJ0YWludHkgZXN0aW1hdGVzIHdoZW4gcHJlZGljdGluZyB0aGUgcHJlc2VuY2Ugb2YgZGlhYmV0aWMgcmV0aW5vcGF0aHkgaW4gcGF0aWVudHMuIFRoaXMgcGFwZXIgZGVtb25zdHJhdGVzIGhvdyB1bmNlcnRhaW50eSBlc3RpbWF0ZXMgY2FuIHByb3ZpZGUgdXNlZnVsIGluZm9ybWF0aW9uIHRvIHRoZSBjbGluaWNpYW4gdGFza2VkIHdpdGggaW50ZXJwcmV0aW5nIHRoZSByZXN1bHRzIG9mIG1lZGljYWwgaW1hZ2VzLiBUaGUga2V5IGlkZWEgaXMgdGhhdCB0aGlzIGh1bWFuLW1hY2hpbmUgaW50ZXJhY3Rpb24gd2lsbCBsZWFkIHRvIG92ZXJhbGwgYmV0dGVyIHJlc3VsdHMgdGhhbiBlaXRoZXIgY291bGQgcHJvZHVjZSBpbmRpdmlkdWFsbHkuCgpJbiB0aGlzIG5vdGVib29rIEkgd2lsbCBmaXJzdCBicmllZmx5IGludHJvZHVjZSB0aGUgZ2VuZXJhbCBpZGVhIGJlaGluZCBNQyBkcm9wb3V0LiBUaGVuIHdlIHdpbGwgYXBwbHkgTUMgZHJvcG91dCBpbiBhIGNsYXNzaWZpY2F0aW9uIHRhc2sgYmFzZWQgb24gdGhlIGNvbGxlY3Rpb24gb2YgbGFiZWxsZWQgaW1hZ2VzIGluIHRoZSBbQ0lGQVItMTAgZGF0YSBzZXRdKGh0dHBzOi8vd3d3LmNzLnRvcm9udG8uZWR1L35rcml6L2NpZmFyLmh0bWwpLgoKIyMgQmFja2dyb3VuZAoKQSBuZXVyYWwgbmV0d29yayBpcyBtYWRlIHVwIG9mIG1hbnkgbmV1cm9ucyB3aGljaCBhcmUgb3JnYW5pemVkIGluIGxheWVycy4gTmV1cm9ucyBhcmUgb2Z0ZW4gY2FsbGVkIG5vZGVzIG9yIHVuaXRzLCBkZXBlbmRpbmcgb24geW91ciBjaG9pY2Ugb2YgbWFjaGluZSBsZWFybmluZyBsaXRlcmF0dXJlLiBIZW5jZWZvcnRoIHdlIHdpbGwgcmVmZXIgdG8gdGhlbSBhcyB1bml0cy4gCgpJbiBhIHR5cGljYWwgbmV1cmFsIG5ldHdvcmsgdGhlcmUgYXJlIG1hbnksIG1hbnkgdW5pdHMuIEFzIGEgY29uc2VxdWVuY2UgdGhlIG51bWJlciBvZiBwYXJhbWV0ZXJzIGdyZWF0bHkgZXhjZWVkcyB0aGUgbnVtYmVyIG9mIGRhdGEgcG9pbnRzLCBkcmFtYXRpY2FsbHkgaW5jcmVhc2luZyB0aGUgcmlzayBvZiBvdmVyZml0dGluZyAoaS5lLiB0aGVyZSBhcmUgbWFueSBzZXR0aW5ncyBvZiB3ZWlnaHRzIHdoaWNoIHdpbGwgY2F1c2UgdGhlIG5ldHdvcmsgdG8gZml0IHRoZSBkYXRhIGFsbW9zdCBwZXJmZWN0bHkpLgoKRHJvcG91dCBpcyBhIGNvbW1vbiBzdG9jaGFzdGljIHJlZ3VsYXJpc2F0aW9uIHRlY2huaXF1ZSBbNV0gdGhhdCBpcyB1c2VkIHRvIHByZXZlbnQgb3ZlcmZpdHRpbmcgaW4gbmV1cmFsIG5ldHdvcmtzLiBUaGUgdGVybSBkcm9wb3V0IHJlZmVycyB0byByYW5kb21seSAiZHJvcHBpbmcgb3V0IiBub25vdXRwdXQgdW5pdHMsIHRlbXBvcmFyaWx5IHJlbW92aW5nIGFsbCBjb25uZWN0aW9ucyB0byB0aGUgcmVzdCBvZiB0aGUgbmV0d29yay4gVGhlIG1haW4gaWRlYSBpcyB0aGF0IGlmIHRoZSBwcmVzZW5jZSBvZiBvdGhlciB1bml0cyBpcyB1bnJlbGlhYmxlLCBlYWNoIHVuaXQgaXMgZm9yY2VkIHRvIGxlYXJuIGhvdyB0byBiZSB1c2VmdWwgb24gaXRzIG93bi4gQXQgdGVzdCB0aW1lIGFsbCB1bml0cyBhcmUgYWN0aXZhdGVkLCBhbmQgdGhlIHdlaWdodHMgb2YgdGhlIG5ldHdvcmsgYXJlIHNjYWxlZCBieSB0aGUgZHJvcG91dCByYXRlIGluIG9yZGVyIHRvIG1hdGNoIHRoZSBleHBlY3RlZCBvdXRwdXQgZHVyaW5nIHRyYWluaW5nLgoKUmVjZW50IHdvcmsgYnkgR2FsIGV0LiBhbC4gWzEsIDIsIDNdIGNhc3RzIGRyb3BvdXQgbmV1cmFsIG5ldHdvcmtzIGFzIGFwcHJveGltYXRlIEJheWVzaWFuIGluZmVyZW5jZS4gVGhlaXIgcmVzdWx0cyBzaG93IHRoYXQgdGhlIHByZWRpY3RpdmUgcG9zdGVyaW9yIGRpc3RyaWJ1dGlvbiBvZiBhIG5ldXJhbCBuZXR3b3JrIGNhbiBiZSBhcHByb3hpbWF0ZWQgYnkgbGVhdmluZyBkcm9wb3V0IG9uIGF0IHRlc3QgdGltZS4gCgpDb25zaWRlciBhIGNsYXNzaWZpY2F0aW9uIHNldHRpbmcsIHN1Y2ggYXMgaW4gQ0lGQVItMTAuIEVzc2VudGlhbGx5LCB3aGF0IGhhcHBlbnMgd2hlbiBhcHBseWluZyBNQyBkcm9wb3V0IGlzIHRoZSBmb2xsb3dpbmc6CgoqIEFuIGltYWdlIGlzIGZlZCBmb3J3YXJkIHRocm91Z2ggdGhlIG5ldHdvcmsgJFQkIHRpbWVzICh3ZSB1c2UgJFQ9MTAwJCBhcyByZWNvbW1lbmRlZCBpbiBbNF0pLiBFYWNoIHRpbWUgdGhlIGltYWdlIGlzIGZlZCB0aHJvdWdoIGlzIGNhbGxlZCBhICpzdG9jaGFzdGljIGZvcndhcmQgcGFzcyouCiogRm9yIGVhY2ggc3RvY2hhc3RpYyBmb3J3YXJkIHBhc3MsIGEgc2xpZ2h0bHkgZGlmZmVyZW50IG5ldHdvcmsgaXMgbWFraW5nIHByZWRpY3Rpb25zIGJlY2F1c2UgZHJvcG91dCBoYXMgcmFuZG9tbHkgc3dpdGNoZWQgb2ZmIHVuaXRzLiAKKiBBcyBhIHJlc3VsdCBlYWNoIHN0b2NoYXN0aWMgZm9yd2FyZCBwYXNzIHJldHVybnMgMTAwIHNsaWdodGx5IGRpZmZlcmVudCB2ZWN0b3JzIG9mIGNsYXNzIHByZWRpY3Rpb25zLgoqIFRvIG1ha2UgYSBwcmVkaWN0aW9uIHdlIGF2ZXJhZ2UgdGhlIDEwMCB2ZWN0b3JzLiBUaGUgY2xhc3MgY29ycmVzcG9uZGluZyB0byB0aGUgbGFyZ2VzdCBlbGVtZW50IGluIHRoZSByZXN1bHRpbmcgdmVjdG9yIGlzIG91ciBmaW5hbCBwcmVkaWN0aW9uLgoqIEZpbmFsbHkgd2UgY2FsY3VsYXRlIHRoZSBzdGFuZGFyZCBkZXZpYXRpb24gaW4gY2xhc3MgcHJlZGljdGlvbnMgb3ZlciBhbGwgZm9yd2FyZCBwYXNzZXMuIFRoaXMgaXMgb3VyIGVzdGltYXRlZCB1bmNlcnRhaW50eS4KCk1hdGhlbWF0aWNhbGx5LCBtb2RlbCB1bmNlcnRhaW55IGlzIGFwcHJveGltYXRlZCB0aGUgZW1waXJpY2FsIHN0YW5kYXJkIGRldmlhdGlvbiBvZiB0aGUgcHJlZGljdGlvbnMgZm9yIGNsYXNzICRrJCwgaS5lLiAkJFxoYXR7XHNpZ21hfV9rID0gXHNxcnR7XGZyYWN7XHN1bV97dD0xfV5Ue1twX3Qoa3xYLHcpIC0gXGhhdHtcbXV9X2t9XV4yfXtULTF9fSQkIHdoZXJlICQkXGhhdHtcbXV9X2sgPSBcZnJhY3sxfXtUfVxzdW1fe3Q9MX1eVCBwX3Qoa3xYLHcpJCQgaXMgdGhlIGF2ZXJhZ2VkIHNvZnRtYXggb3V0cHV0cyBvZiB0aGUgcHJlZGljdGVkIGNsYXNzLgoKR2FsIGV0LiBhbC4gWzNdIHNob3cgdGhhdCB0aGUgYWJvdmUgYW1vdW50cyB0byBkcmF3aW5nIE1vbnRlIENhcmxvIHNhbXBsZXMgZnJvbSB0aGUgcHJlZGljdGl2ZSBwb3N0ZXJpb3IuIFRoZWlyIHdvcmsgZGVtb25zdHJhdGVzIHRoYXQgYXBwbHlpbmcgZHJvcG91dCBpcyBlZmZlY3RpdmVseSB0aGUgc2FtZSBhcyBkZWZpbmluZyBhICpCYXllc2lhbiBuZXVyYWwgbmV0d29yayogd2l0aCBhIEJlcm5vdWxsaSBhcHByb3hpbWF0aW5nIHByaW9yIG92ZXIgdGhlIHBhcmFtZXRlcnMuIFtHYWwgaGFzIHdyaXR0ZW4gYW4gZXhjZWxsZW50IGJsb2cgcG9zdCB0aGF0IGludHJvZHVjZXMgdGhlIHdvcmtdKGh0dHA6Ly9tbGcuZW5nLmNhbS5hYy51ay95YXJpbi9ibG9nXzNkODAxYWE1MzJjMWNlLmh0bWwpIGFuZCB0aGUgZGVyaXZhdGlvbi4KCiMjIFByb2JsZW0gc3RhdGVtZW50CgpUaGUgZGVyaXZhdGlvbiBpbiBHYWwgZXQuIGFsLiBbMl0gaXMgYmFzZWQgb24gdGhlIG9ic2VydmF0aW9uIHRoYXQgYSBzaGFsbG93IG5ldXJhbCBuZXR3b3JrIHdpdGggaW5maW5pdGVseSBtYW55IHdlaWdodHMgY29udmVyZ2VzIHRvIGEgR2F1c3NpYW4gcHJvY2VzcyAoR1ApIHdpdGggYSBzcGVjaWZpYyBjb3ZhcmlhbmNlIGZ1bmN0aW9uLiBBIEdQIGlzIGEgbm9uLXBhcmFtZXRyaWMsIHByb2JhYmlsaXN0aWMgbWFjaGluZSBsZWFybmluZyBtZXRob2QuIFRoZSBpZGVhIGlzIHRoYXQgd2UgcGxhY2UgYSBwcmlvciBkaXN0cmlidXRpb24gb3ZlciB0aGUgZnVuY3Rpb24gc3BhY2UsIGFuZCBieSBvYnNlcnZpbmcgbmV3IGRhdGEgcG9pbnRzIHdlIGNhbiBmaWd1cmUgb3V0IHdoaWNoIGZ1bmN0aW9uIGlzIG1vc3QgbGlrZWx5IHRvIGhhdmUgZ2VuZXJhdGVkIHRoZSBvYnNlcnZlZCBkYXRhLiBBIEdQIGlzIGZ1bGx5IHNwZWNpZmllZCBieSBpdHMgbWVhbiBhbmQgY292YXJpYW5jZSBmdW5jdGlvbnMsIGFuZCBhbGxvd3MgdXMgdG8gb2J0YWluIHVuY2VydGFpbnR5IGVzdGltYXRlcyBbMV0uIFRoZSBleHRlbnNpb24gb2YgdGhpcyB0byBkcm9wb3V0IG5ldXJhbCBuZXR3b3JrcyBpcyB0aGUgbWFpbiByZXN1bHQgb2YgR2FsIGV0LiBhbC4gWzJdLgoKSW4gWzFdIGhvd2V2ZXIsIEdhbCBldC4gYWwuIGV4dGVuZCB0aGlzIGlkZWEgZnVydGhlciB0byBpbmNsdWRlIHByaW9ycyBvdmVyIHRoZSB3ZWlnaHRzIG9mIHRoZSBjb252b2x1dGlvbmFsIGxheWVycyBpbiBhIGNvbnZvbHV0aW9uYWwgbmV1cmFsIG5ldHdvcmsgKENOTikuIEEgQ05OIGlzIGEgc3BlY2lhbGl6ZWQgdHlwZSBvZiBuZXVyYWwgbmV0d29yaywgdXNlZCBwcmltYXJpbHkgYW5kIHZlcnkgZWZmZWN0aXZlbHkgZm9yIGltYWdlIGFuYWx5c2lzLiBUaGUgYXV0aG9ycyBicmllZmx5IHN0YXRlIHRoYXQgdGhlIEdQIGludGVycHJldGF0aW9uIGlzIGxvc3Qgd2hlbiB0aGUgbW9kZWwgaXMgZXh0ZW5kZWQgdG8gQ05OcywgYnV0IHRoYXQgdGhlc2UgbmV0d29ya3Mgc3RpbGwgY2FuIGJlICJtb2RlbGxlZCBhcyBCYXllc2lhbiIuCgpBcyBmYXIgYXMgd2UgYXJlIGF3YXJlLCB0aGVyZSBoYXZlIGJlZW4gbm8gZWZmb3J0cyB0byBkZXRlcm1pbmUgdGhlIGNvcnJlbGF0aW9uIGJldHdlZW4gdGhlIGFwcHJveGltYXRlZCB1bmNlcnRhaW50eSBhbmQgYSBuZXR3b3JrJ3MgYWJpbGl0eSB0byBwcmVkaWN0IGNvcnJlY3RseS4gTW9yZW92ZXIsIHRoZSB3b3JrIGRvbmUgc28gZmFyIHNlZW1zIHRvIGZvY3VzIG9uIHRoZSB1bmNlcnRhaW50eSBhc3NvY2lhdGVkIHdpdGggdGhlIHByZWRpY3RlZCBjbGFzcy4gV2Ugd2lsbCBleGFtaW5lIGlmIHRoZXJlIGlzIGFueSBhZGRpdGlvbmFsIGluZm9ybWF0aW9uIHRvIGJlIGdhaW5lZCBmcm9tIGVzdGFibGlzaGluZyBhIGNvbm5lY3Rpb24gYmV0d2VlbiB0aGUgcHJlZGljdGlvbiBhbmQgdGhlICpydW5uZXItdXAqIHByZWRpY3Rpb24uIEluIG90aGVyIHdvcmRzLCBvdXIgcHJvYmxlbSBzdGF0ZW1lbnQgaXM6Cgo+ICpBcmUgdGhlIHVuY2VydGFpbnR5IGFwcHJveGltYXRpb25zIG9idGFpbmVkIGJ5IGFwcGx5aW5nIE1vbnRlIENhcmxvIGRyb3BvdXQgdG8gY29udm9sdXRpb25hbCBuZXVyYWwgbmV0d29ya3MgYSByZWFzb25hYmxlIG1lYXN1cmUgb2YgbW9kZWwgdW5jZXJ0YWludHk/KgoKIyMgRXhwZXJpbWVudGFsIHNldHVwey50YWJzZXR9CgpTdGF0ZS1vZi10aGUtYXJ0IGFyY2hpdGVjdHVyZXMgc3VjaCBhcyBSZXNOZXQgYW5kIERlbnNlTmV0IGFyZSB2ZXJ5IHBvd2VyZnVsLCBidXQgdGhleSBhcmUgYWxzbyBjb21wbGljYXRlZCBhbmQgdGhlaXIgaW5uZXIgd29ya2luZ3MgYXJlIHF1aXRlIGNvbnZvbHV0ZWQuIFdlIGFyZSBwcmltYXJpbHkgaW50ZXJlc3RlZCBpbiBleGFtaW5pbmcgdGhlIGNvcnJlbGF0aW9uIGJldHdlZW4gdW5jZXJ0YWludHkgZXN0aW1hdGlvbiBhbmQgcHJlZGljdGl2ZSBjYXBhYmlsaXRpZXMuIEl0IGlzIGFyZ3VhYmx5IGJldHRlciB0byB1c2UgYSBzaW1wbGUgbmV0d29yayBhcmNoaXRlY3R1cmUgdG8gaWxsdXN0cmF0ZSB0aGUgaWRlYSBvZiBNQyBkcm9wb3V0LiBJbiBvdXIgYXBwcm9hY2ggd2UgdXNlIExlTmV0LTUuIENsaWNrIG9uIHRoZSB0YWJzIGZvciBpbXBsZW1lbnRhdGlvbiBkZXRhaWxzLiBGdWxsIGNvZGUgaXMgYXZhaWxhYmxlIG9uIFtHaXRIdWJdKGh0dHBzOi8vZ2l0aHViLmNvbS9zbXUwOTUvc211MDk1LmdpdGh1Yi5pby9ibG9iL21hc3Rlci9kYXQyNTkuaXB5bmIpLgoKIyMjIE1vZGVsCgpMZU5ldC01IHdhcyBhIHBpb25lZXJpbmcgNy1sYXllciBjb252b2x1dGlvbmFsIG5ldXJhbCBuZXR3b3JrIG9yaWdpbmFsbHkgZGV2ZWxvcGVkIGJ5IFlhbm4gTGVDdW5uIGluIDE5OTggZm9yIGhhbmR3cml0dGVuIGRpZ2l0IHJlY29nbml0aW9uLiBJdCBpcyBob3BlbGVzc2x5IHByaW1pdGl2ZSBjb21wYXJlZCB0byBjb250ZW1wb3JhcnkgYXJjaGl0ZWN0dXJlcywgYnV0IHN0aWxsIGNhcHR1cmVzIHRoZSBnaXN0IG9mIHdoYXQgYSBjb252b2x1dGlvbmFsIG5ldHdvcmsgaXMgd2hpbGUgcmVtYWluaW5nIHNpbXBsZSBlbm91Z2ggdG8gYWxsb3cgdXMgdG8gdW5kZXJzdGFuZCBldmVyeSBidWlsZGluZyBibG9jayBvZiB0aGUgbmV0d29yay4gVGhlIGZvbGxvd2luZyBjaHVuayBzaG93cyB0aGUgbW9kZWwgYXJjaGl0ZWN0dXJlLiBUb2dnbGUgYENvZGVgIGJ1dHRvbiB0byB0aGUgcmlnaHQgdG8gdmlldy4KCmBgYHtweXRob24gcHl0aG9uLnJldGljdWxhdGU9RkFMU0UsIGV2YWw9RkFMU0V9CiMgQnVpbGRpbmcgc2ltcGxlIEJheWVzaWFuIENOTiB3aXRoIE1DIGRyb3BvdXQgYW5kIFNHRCBvcHRpbWl6ZXIKZGVmIGxlbmV0X2FsbChpbnB1dF9zaGFwZSwgbmJfY2xhc3NlcywgcD0wLjUpOgoKICAgICMgQnVpbGRpbmcgbW9kZWwKICAgIG1vZGVsPVNlcXVlbnRpYWwoWwogICAgICAgIENvbnYyRChpbnB1dF9zaGFwZT1pbnB1dF9zaGFwZSwgZmlsdGVycz0xOTIsIGtlcm5lbF9zaXplPSg1LDUpKSwgIyAyRCBjb252b2x1dGlvbiBsYXllciwgZS5nLiBzcGF0aWFsIGNvbnZvbHV0aW9uIG92ZXIgaW1hZ2VzLgogICAgICAgIERyb3BvdXRNQyhwKSwKICAgICAgICBNYXhQb29saW5nMkQoc3RyaWRlcz0yKSwgIyBNYXggcG9vbGluZyBvcGVyYXRpb24gZm9yIHNwYXRpYWwgZGF0YQogICAgICAgICAgICAgICAgCiAgICAgICAgQ29udjJEKDE5Miwga2VybmVsX3NpemU9KDUsNSkpLCAjIDJEIGNvbnZvbHV0aW9uIGxheWVyLCBlLmcuIHNwYXRpYWwgY29udm9sdXRpb24gb3ZlciBpbWFnZXMuCiAgICAgICAgRHJvcG91dE1DKHApLAogICAgICAgIE1heFBvb2xpbmcyRChzdHJpZGVzPTIpLCAjIE1heCBwb29saW5nIG9wZXJhdGlvbiBmb3Igc3BhdGlhbCBkYXRhCiAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgRmxhdHRlbigpLCAjIEZsYXR0ZW5zIHRoZSBpbnB1dAogICAgICAgIERlbnNlKDEwMDAsIGFjdGl2YXRpb249InJlbHUiKSwgIyBGdWxseSBjb25uZWN0ZWQgbGF5ZXIKICAgICAgICBEcm9wb3V0TUMocCksCiAgICAgICAgRGVuc2UobmJfY2xhc3NlcywgYWN0aXZhdGlvbj0ic29mdG1heCIpICMgT3V0cHV0IGxheWVyCiAgICBdKSAKICAgIAogICAgIyBDb21waWxpbmcgbW9kZWwKICAgIG1vZGVsLmNvbXBpbGUoU0dEKG1vbWVudHVtPTAuOSwgZGVjYXk9MC4wMDA1KSwgIyBPcHRpbWlzZXIKICAgICAgICAgICAgICAgICAgbG9zcz0iY2F0ZWdvcmljYWxfY3Jvc3NlbnRyb3B5IiwgIyBNaW5pbWlzYXRpb24gb2JqZWN0aXZlCiAgICAgICAgICAgICAgICAgIG1ldHJpY3M9WyJhY2N1cmFjeSJdKSAjIEV2YWx1YXRpb24gbWV0cmljCiAgICAKICAgIHJldHVybiBtb2RlbApgYGAKPGJyPgoKIyMjIE1DIGRyb3BvdXQgbGF5ZXIKCk91ciBzcGVjaWZpY2F0aW9uIG9mIExlTmV0LTUgZGlmZmVycyBmcm9tIHRoZSBvcmdpbmlhbCBpbiBvbmUgY3J1Y2lhbCB3YXk6IFdlIHVzZSBNb250ZSBDYXJsbyBkcm9wb3V0IGxheWVycy4gTUMgZHJvcG91dCBpcyBub3QgYSBmZWF0dXJlIHRoYXQgaXMgaW1wbGVtZW50ZWQgaW4gUHlUb3JjaCwgYW5kIHdlIG11c3QgdGhlcmVmb3JlIGltcGxlbWVuZXQgb25lIG91cnNlbHZlcy4gRm9ydHVuYXRlbHksIHRoaXMgYW1vdW50cyB0byBhIHNpbXBsZSBhZGp1c3RtZW50IG9mIGV4aXN0aW5nIGNvZGUuIFdlIG1vZGlmeSB0aGUgYERyb3BvdXRgIGNsYXNzIHRvIHRha2UgYW4gYWRkaXRpb25hbCBhcmd1bWVudCBjYWxsZWQgYGRyb3BvdXRNQ2Agd2l0aCBkZWZhdWx0IHZhbHVlIHNldCB0byBgVHJ1ZWAuIFRvZ2dsZSBgQ29kZWAgYnV0dG9uIHRvIHRoZSByaWdodCB0byB2aWV3LgoKYGBge3B5dGhvbiBweXRob24ucmV0aWN1bGF0ZT1GQUxTRSwgZXZhbD1GQUxTRX0KY2xhc3MgRHJvcG91dE1DKExheWVyKToKICAgICIiIkFwcGxpZXMgTUMgRHJvcG91dCB0byB0aGUgaW5wdXQuCiAgICBEcm9wb3V0IGNvbnNpc3RzIGluIHJhbmRvbWx5IHNldHRpbmcKICAgIGEgZnJhY3Rpb24gYHJhdGVgIG9mIGlucHV0IHVuaXRzIHRvIDAgYXQgZWFjaCB1cGRhdGUgZHVyaW5nIHRyYWluaW5nIHRpbWUsCiAgICB3aGljaCBoZWxwcyBwcmV2ZW50IG92ZXJmaXR0aW5nLgogICAgTUMgRHJvcG91dCBjb25zaXN0cyBpbiBzYW1wbGluZyBmcm9tIHRoZSBwb3N0ZXJpb3IgcHJlZGljdGl2ZSBkaXN0cmlidXRpb24gYnkgYWN0aXZhdGluZyBkcm9wb3V0IGF0IHRlc3QgdGltZS4KICAgICMgQXJndW1lbnRzCiAgICAgICAgcmF0ZTogZmxvYXQgYmV0d2VlbiAwIGFuZCAxLiBGcmFjdGlvbiBvZiB0aGUgaW5wdXQgdW5pdHMgdG8gZHJvcC4KICAgICAgICBub2lzZV9zaGFwZTogMUQgaW50ZWdlciB0ZW5zb3IgcmVwcmVzZW50aW5nIHRoZSBzaGFwZSBvZiB0aGUKICAgICAgICAgICAgYmluYXJ5IGRyb3BvdXQgbWFzayB0aGF0IHdpbGwgYmUgbXVsdGlwbGllZCB3aXRoIHRoZSBpbnB1dC4KICAgICAgICAgICAgRm9yIGluc3RhbmNlLCBpZiB5b3VyIGlucHV0cyBoYXZlIHNoYXBlCiAgICAgICAgICAgIGAoYmF0Y2hfc2l6ZSwgdGltZXN0ZXBzLCBmZWF0dXJlcylgIGFuZAogICAgICAgICAgICB5b3Ugd2FudCB0aGUgZHJvcG91dCBtYXNrIHRvIGJlIHRoZSBzYW1lIGZvciBhbGwgdGltZXN0ZXBzLAogICAgICAgICAgICB5b3UgY2FuIHVzZSBgbm9pc2Vfc2hhcGU9KGJhdGNoX3NpemUsIDEsIGZlYXR1cmVzKWAuCiAgICAgICAgc2VlZDogQSBQeXRob24gaW50ZWdlciB0byB1c2UgYXMgcmFuZG9tIHNlZWQuCiAgICAjIFJlZmVyZW5jZXMKICAgICAgICAtIFtEcm9wb3V0OiBBIFNpbXBsZSBXYXkgdG8gUHJldmVudCBOZXVyYWwgTmV0d29ya3MgZnJvbSBPdmVyZml0dGluZ10oaHR0cDovL3d3dy5jcy50b3JvbnRvLmVkdS9+cnNhbGFraHUvcGFwZXJzL3NyaXZhc3RhdmExNGEucGRmKQogICAgIiIiCiAgICBAaW50ZXJmYWNlcy5sZWdhY3lfZHJvcG91dF9zdXBwb3J0CiAgICBkZWYgX19pbml0X18oc2VsZiwgcmF0ZSwgbm9pc2Vfc2hhcGU9Tm9uZSwgc2VlZD1Ob25lLCAqKmt3YXJncyk6CiAgICAgICAgc3VwZXIoRHJvcG91dE1DLCBzZWxmKS5fX2luaXRfXygqKmt3YXJncykKICAgICAgICBzZWxmLnJhdGUgPSBtaW4oMS4sIG1heCgwLiwgcmF0ZSkpCiAgICAgICAgc2VsZi5ub2lzZV9zaGFwZSA9IG5vaXNlX3NoYXBlCiAgICAgICAgc2VsZi5zZWVkID0gc2VlZAogICAgICAgIHNlbGYuc3VwcG9ydHNfbWFza2luZyA9IFRydWUKCiAgICBkZWYgX2dldF9ub2lzZV9zaGFwZShzZWxmLCBpbnB1dHMpOgogICAgICAgIGlmIHNlbGYubm9pc2Vfc2hhcGUgaXMgTm9uZToKICAgICAgICAgICAgcmV0dXJuIHNlbGYubm9pc2Vfc2hhcGUKCiAgICAgICAgc3ltYm9saWNfc2hhcGUgPSBLLnNoYXBlKGlucHV0cykKICAgICAgICBub2lzZV9zaGFwZSA9IFtzeW1ib2xpY19zaGFwZVtheGlzXSBpZiBzaGFwZSBpcyBOb25lIGVsc2Ugc2hhcGUKICAgICAgICAgICAgICAgICAgICAgICBmb3IgYXhpcywgc2hhcGUgaW4gZW51bWVyYXRlKHNlbGYubm9pc2Vfc2hhcGUpXQogICAgICAgIHJldHVybiB0dXBsZShub2lzZV9zaGFwZSkKCiAgICBkZWYgY2FsbChzZWxmLCBpbnB1dHMsIHRyYWluaW5nPVRydWUpOgogICAgICAgIGlmIDAuIDwgc2VsZi5yYXRlIDwgMS46CiAgICAgICAgICAgIG5vaXNlX3NoYXBlID0gc2VsZi5fZ2V0X25vaXNlX3NoYXBlKGlucHV0cykKCiAgICAgICAgICAgIGRlZiBkcm9wcGVkX2lucHV0cygpOgogICAgICAgICAgICAgICAgcmV0dXJuIEsuZHJvcG91dChpbnB1dHMsIHNlbGYucmF0ZSwgbm9pc2Vfc2hhcGUsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHNlZWQ9c2VsZi5zZWVkKQogICAgICAgICAgICByZXR1cm4gSy5pbl90cmFpbl9waGFzZShkcm9wcGVkX2lucHV0cywgaW5wdXRzLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0cmFpbmluZz10cmFpbmluZykKICAgICAgICByZXR1cm4gaW5wdXRzCgogICAgZGVmIGdldF9jb25maWcoc2VsZik6CiAgICAgICAgY29uZmlnID0geydyYXRlJzogc2VsZi5yYXRlLAogICAgICAgICAgICAgICAgICAnbm9pc2Vfc2hhcGUnOiBzZWxmLm5vaXNlX3NoYXBlLAogICAgICAgICAgICAgICAgICAnc2VlZCc6IHNlbGYuc2VlZH0KICAgICAgICBiYXNlX2NvbmZpZyA9IHN1cGVyKERyb3BvdXRNQywgc2VsZikuZ2V0X2NvbmZpZygpCiAgICAgICAgcmV0dXJuIGRpY3QobGlzdChiYXNlX2NvbmZpZy5pdGVtcygpKSArIGxpc3QoY29uZmlnLml0ZW1zKCkpKQoKICAgIGRlZiBjb21wdXRlX291dHB1dF9zaGFwZShzZWxmLCBpbnB1dF9zaGFwZSk6CiAgICAgICAgcmV0dXJuIGlucHV0X3NoYXBlCmBgYAo8YnI+CgojIyMgSW5mZXJlbmNlCgpXZSBuZWVkIHRvIGRlZmluZSBhIGZ1bmN0aW9uIHRoYXQgcGVyZm9ybXMgaW5mZXJlbmNlIG92ZXIgb3V0IGlucHV0LiBUaGUgZm9sbG93aW5nIGNodW5rIGNvbnRhaW5zIHRoZSByZWxldmFudCBjb2RlIGZvciB0aGUgc2FtcGxpbmcgcHJvY2VkdXJlIGRlc2NyaWJlZCBpbiBzZWN0aW9uIDEuMS4gVGhlIGZ1bmN0aW9uIGBpbmZlcmVuY2VgIHN0b3JlcyBhbGwgdGhlIHJlbGV2YW50IHN0YXRpc3RpY3MgYW5kIHNvZnRtYXggZGlzdHJpYnV0aW9ucyBpbiBhIGRpY3Rpb25hcnkgbmFtZWQgYG91dHB1dGAuIFRoZSByZXN1bHRzIGFyZSB0aGVuIHR1cm5lZCBpbnRvIGEgcGFuZGFzIGRhdGFmcmFtZSBhbmQgc29tZSB2ZXJ5IGJhc2ljIGZlYXR1cmUgZW5naW5lZXJpbmcgaXMgcGVyZm9ybWVkLiBGaW5hbGx5LCB0aGUgZGF0YSBpcyBwcmVwYXJlZCBmb3Igc3RhdGlzdGljYWwgYW5hbHlzaXMgaW4gUi4gVG9nZ2xlIGBDb2RlYCBidXR0b24gdG8gdGhlIHJpZ2h0IHRvIHZpZXcuCgpgYGB7cHl0aG9uIHB5dGhvbi5yZXRpY3VsYXRlPUZBTFNFLCBldmFsPUZBTFNFfQpkZWYgYmF0Y2goaW1nLCBUPTEwMCk6CiAgICAnJycgQ3JlYXRpbmcgbWluaS1iYXRjaCBvZiBUIGlkZW50aWNhbCBpbWFnZXMgZm9yIHVzZSBpbiBpbmZlcmVuY2UuCiAgICAgICAgCiAgICAgICAgQXJndW1lbnRzOgogICAgICAgIGltZywgbnVtcHkgYXJyYXkKICAgICAgICBULCBudW1iZXIgb2Ygc3RvY2hhc3RpYyBmb3J3YXJkIHBhc3NlcyAoaS5lLiBudW1iZXIgb2YgdGltZXMgaW1hZ2UgbmVlZHMgdG8gYmUgcmVwZWF0ZWQpCiAgICAgICAgJycnCiAgICBpbWdfYmF0Y2ggPSBbXQogICAgCiAgICBmb3IgbiBpbiByYW5nZSgwLFQpOgogICAgICAgIGltZ19iYXRjaC5hcHBlbmQoaW1nKQoKICAgIHJldHVybihucC5hcnJheShpbWdfYmF0Y2gpKQoKZGVmIGluZmVyZW5jZShtb2RlbCwgWCwgeSwgVD0xMDAsIG5vcm1hbGl6ZT1GYWxzZSk6ICAgIAogICAgJycnIEZ1bmN0aW9uIHRoYXQgcGVyZm9ybXMgaW5mZXJlbmNlIGJ5IGFwcGx5aW5nIE1DIGRyb3BvdXQgd2l0aCBUIHN0b2NoYXN0aWMgZm9yd2FyZCBwYXNzZXMgb24gZ2l2ZW4gaW5wdXQuCiAgICAgICAgCiAgICAgICAgQXJndW1lbnRzOgogICAgICAgIG1vZGVsLCBhIG1vZGVsIG9iamVjdAogICAgICAgIFgsIGltYWdlcyB0byBiZSBjbGFzc2lmaWVkCiAgICAgICAgeSwgY29ycmVzcG9uZGluZyBsYWJlbHMsCiAgICAgICAgVCwgbnVtYmVyIG9mIHN0b2NoYXN0aWMgZm9yd2FyZCBwYXNzZXMKICAgICAgICAnJycKICAgICMgR2V0IGltYWdlcywgbGFiZWxzCiAgICBpbWdzLCBsYWJlbHMgPSBYLCB5LmFyZ21heChheGlzPTEpCiAgICAKICAgIGlmIG5vcm1hbGl6ZToKICAgICAgICAjIFByZXByb2Nlc3NpbmcgaW5wdXQKICAgICAgICBpbWdzID0gaW1ncy5hc3R5cGUoImZsb2F0MzIiKQogICAgICAgIGltZ3MgLT0gbnAubWVhbihYKQogICAgICAgIGltZ3MgLz0gbnAuc3RkKFgsIGF4aXM9MCkKICAgIAogICAgIyBFbXB0eSBkaWN0aW9uYXJ5IHRvIHN0b3JlIGFsbCBvdXRwdXQKICAgIG91dHB1dCA9IHt9CgogICAgaz0wICMgaXRlcmF0b3IgaW5kZXggdG8ga2VlcCBpbiBkaWN0aW9uYXJ5CgogICAgZm9yIChpbWcsIGxhYmVsLCB4KSBpbiBsaXN0KHppcChpbWdzLCBsYWJlbHMsIFgpKTogI2FkZGVkIHgsWAogICAgICAgIAogICAgICAgICMgR2V0IGltYWdlIGJhdGNoCiAgICAgICAgaW1nX2JhdGNoID0gYmF0Y2goaW1nLCBUKQogICAgICAgIAogICAgICAgICMgVCBwcmVkaWN0aW9ucyBvbiBzYW1lIGltYWdlCiAgICAgICAgcmVzdWx0cyA9IG1vZGVsLnByZWRpY3QoaW1nX2JhdGNoLCBiYXRjaF9zaXplPVQpCgogICAgICAgICMgR2F0aGVyaW5nIHJlc3VsdHMKICAgICAgICBwcm9icyA9IHJlc3VsdHMKICAgICAgICBwcm9ic19tZWFuID0gbnAubWVhbihwcm9icywgYXhpcz0wKQogICAgICAgIHByZWRfc3RkID0gbnAuc3RkKHByb2JzLCBheGlzPTApCiAgICAgICAgcHJlZGljdGlvbiA9IHByb2JzX21lYW4uYXJnbWF4KCkKICAgICAgICB1bmNlcnRhaW50eSA9IHByZWRfc3RkW3ByZWRpY3Rpb25dCiAgICAgICAgY29ycmVjdCA9IDEgaWYgcHJlZGljdGlvbiA9PSBsYWJlbCBlbHNlIDAKCiAgICAgICAgb3V0cHV0W2tdID0geyJpbWciOiB4LCAic29mdG1heF9kaXN0IjogcHJvYnMsICJwcm9icyI6IHByb2JzX21lYW4sICJwcmVkaWN0aW9uIjogcHJlZGljdGlvbiwgInRydXRoIjogbGFiZWwsICJ1bmNlcnRhaW50eSI6IHVuY2VydGFpbnR5LCAiY29ycmVjdCI6IGNvcnJlY3R9ICMgYWRkZWQgeCBmb3IgaW1nCiAgICAKICAgICAgICBrKz0xCiAgICAgICAgCiAgICByZXR1cm4gb3V0cHV0CmBgYAoKIyMgTW9kZWxzCgpXZSB3aWxsIGV4YW1pbmUgZGF0YSBnYXRoZXJlZCBmcm9tIGZvdXIgdmFyaWFudHMgb2YgTGVOZXQtNS4gQWxsIG1vZGVscyB3ZXJlIHRyYWluZWQgb24gQ0lGQVItMTAgdXNpbmcgdGhlIGBmYXN0YWlgIEFQSS4gQ0lGQVItMTAgY29udGFpbnMgNjAuMDAwIGxhYmVsbGVkIDMyeDMyeDMgY29sb3IgaW1hZ2VzIGJlbG9uZ2luZyB0byAxMCBkaWZmZXJlbnQgY2xhc3Nlcy4gVGhlIGlucHV0IGRhdGEgd2FzIHNwbGl0IGludG8gYSB0cmFpbmluZyBzZXQgb2YgNTAuMDAwIGltYWdlcyBhbmQgYSB0ZXN0IHNldCBvZiAxMC4wMDAgaW1hZ2VzLiBUaGUgdHJhaW5pbmcgc2V0IHdhcyBmdXJ0aGVyIHNwbGl0IGludG8gYSB0cmFpbmluZyBzZXQgYW5kIGEgdmFsaWRhdGlvbiBzZXQuIEFsbCBtb2RlbHMgaGF2ZSBgd2VpZ2h0X2RlY2F5ID0gMC4wMDA1YCBhbmQgYWxsIGxlYXJuaW5nIHJhdGVzIHdlcmUgY2hvc2VuIHVzaW5nIGBscl9maW5kYDoKCiogYG1vZGVsNTVgOiBUcmFpbmVkIGZvciA2MCBlcG9jaHMgd2l0aCBhIGxlYXJuaW5nIHJhdGUgb2YgMC4wMDEgYW5kIGBrZXJuZWwgc2l6ZSA9ICg1LDUpYCBhbmQgYGRyb3BfcmF0ZSA9IC41YC4gICoqMC43MTI4MCB2YWxpZGF0aW9uIGxvc3MqKiBhdCBlbmQgb2YgdHJhaW5pbmcgd2l0aCBhbiAqKmFjY3VyYWN5IG9mIDAuNzYxMzcqKiBvbiB0aGUgdmFsaWRhdGlvbiBkYXRhLiBgbW9kZWw1NWAgcmVwcmVzZW50cyB0aGUgYmFzZWxpbmUgaW1wbGVtZW50YXRpb24gb2YgTGVOZXQtNS4gSXQgaXMgaWRlbnRpY2FsIGluIHN0cnVjdHVyZSB0byB0aGUgb25lIHVzZWQgYnkgR2FsIGV0LiBhbC4gWzFdLgoKKiBgbW9kZWw1MmA6IFRyYWluZWQgZm9yIDE0IGVwb2NocyB3aXRoIGEgbGVhcm5pbmcgcmF0ZSBvZiAwLjAxIGZvciB0aGUgZmlyc3QgNyBhbmQgMC4wMDAxIG9uIHRoZSByZW1haW5pbmcuIENoYW5nZWQgZHVlIHRvIHJhcGlkIG92ZXJmaXR0aW5nLiBNb2RlbCBoYXMgYGtlcm5lbCBzaXplID0gKDUsNSlgIGFuZCBgZHJvcF9yYXRlID0gLjJgLiAqKjAuNzQxODYgdmFsaWRhdGlvbiBsb3NzKiogYXQgZW5kIG9mIHRyYWluaW5nIHdpdGggYW4gKiphY2N1cmFjeSBvZiAwLjc0NDA2Kiogb24gdGhlIHZhbGlkYXRpb24gZGF0YS4KCiogYG1vZGVsMzVgOiBUcmFpbmVkIGZvciA2NiBlcG9jaHMgd2l0aCBhIGxlYXJuaW5nIHJhdGUgb2YgMC4wMDEgYW5kIGBrZXJuZWwgc2l6ZSA9ICgzLDMpYCBhbmQgYGRyb3BfcmF0ZSA9IC41YC4gKiowLjc1NDgzIHZhbGlkYXRpb24gbG9zcyoqIGF0IGVuZCBvZiB0cmFpbmluZyB3aXRoIGFuICoqYWNjdXJhY3kgb2YgMC43NDIxOCoqIG9uIHRoZSB2YWxpZGF0aW9uIGRhdGEuIAoKKiBgbW9kZWwzMmA6IFRyYWluZWQgZm9yIDUyIGVwb2NocyB3aXRoIGEgbGVhcm5pbmcgcmF0ZSBvZiAwLjAwMSBhbmQgYGtlcm5lbCBzaXplID0gKDMsMylgIGFuZCBgZHJvcF9yYXRlID0gLjJgLiAqKjAuNzU0NDkgdmFsaWRhdGlvbiBsb3NzKiogYXQgZW5kIG9mIHRyYWluaW5nIHdpdGggYW4gKiphY2N1cmFjeSBvZiAwLjc0MDkwKiogb24gdGhlIHZhbGlkYXRpb24gZGF0YS4KCk5vdGUgdGhhdCBgbW9kZWw1NWAgaGFzIGEgc2xpZ2h0bHkgYmV0dGVyIGJhc2VsaW5lIHBlcmZvcm1hbmNlIHRoYW4gdGhlIG90aGVyIG1vZGVscy4KCgojIyBEYXRhCgpUaGUgZGF0YSBjb250YWlucyB0aGUgZm9sbG93aW5nIHZhcmlhYmxlcyBhZnRlciBpdCBoYXMgYmVlbiBwcmVwYXJlZCBmb3IgYW5hbHlzaXMgaW4gUjoKCiogYGNvcnJlY3RgIChsb2dpY2FsKTogaW5kaWNhdG9yIHRoZSBpcyBgVFJVRWAgaWYgdGhlIHByZWRpY3RlZCBjbGFzcyBsYWJlbCBtYXRjaGVzIHRoZSB0cnVlIGNsYXNzIGxhYmVsLCBlbHNlIGBGQUxTRWAuCgoqIGBwcmVkaWN0aW9uYCAoaW50KTogcHJlZGljdGVkIGNsYXNzIGxhYmVsLgoKKiBgdHJ1dGhgIChpbnQpOiB0cnVlIGNsYXNzIGxhYmVsLgoKKiBgdW5jZXJ0YWludHlgIChkYmwpOiBlbXBpcmljYWwgc3RhbmRhcmQgZGV2aWF0aW9uIG9mIHNvZnRtYXggdmFsdWVzIGZvciBwcmVkaWN0ZWQgY2xhc3MuCgoqIGBwcm9iMWAgKGRibCk6IGFyZ21heCBvZiBtZWFuIHNvZnRtYXggb3V0cHV0LCBpLmUuIG1lYW4gcHJvYmFiaWxpdHkgb2YgcHJlZGljdGVkIGNsYXNzLgoKKiBgcHJvYjJgIChkYmwpOiBtZWFuIHByb2JhYmlsaXR5IG9mIHJ1bm5lci11cCBwcmVkaWN0aW9uLgoKKiBgY2xhc3MyYCAoaW50KTogY2xhc3MgbGFiZWwgb2YgcnVubmVyLXVwIHByZWRpY3Rpb24uCgoqIGBsb2dpdF9wcm9iMWAgKGRibCk6IGxvZ2l0IHRyYW5zZm9ybWF0aW9uIG9mIGBwcm9iMWAuCgoqIGBkaWZmYCAoZGJsKTogYHByb2IxYC1gcHJvYjJgCgoqIGBkaWZmX3NkX3JhdGlvYCAoZGJsKTogYGRpZmYvdW5jZXJ0YWludHlgLgoKQWxsIHRoZSB2YXJpYWJsZXMgYWJvdmUgYXJlIHByZXR0eSBzdGFuZGFyZCwgd2l0aCB0aGUgZXhjZXB0aW9uIG9mIGBkaWZmX3NkX3JhdGlvYC4gSW50dWl0aXZlbHksIGlmIGBkaWZmYCBpcyBsYXJnZSwgdGhlIGF2ZXJhZ2VkIG1vZGVscyBhbGwgYWdyZWUgdGhhdCBjbGFzcyAkayQgaXMgdGhlIGNvcnJlY3QgcHJlZGljdGlvbi4gSWYgYGRpZmZgIGlzIHNtYWxsLCB0aGUgbW9kZWxzIHNhbXBsZWQgYnkgTUMgZHJvcG91dCBkb24ndCBhZ3JlZSBvbiBhIHNpbmdsZSBjbGFzcy4gVGh1cyBgZGlmZmAgYWxzbyBzZXJ2ZXMgYXMgYSBwcm94eSBmb3IgdW5jZXJ0YWludHkuIE1vZGVsIGB1bmNlcnRhaW50eWAsIGhvd2V2ZXIsIGlzIGFwcHJveGltYXRlZCBieSB0aGUgZW1waXJpY2FsIHN0YW5kYXJkIGRldmlhdGlvbiBvZiB0aGUgcHJlZGljdGlvbnMgZm9yIGNsYXNzICRrJC4gVGh1cyBgZGlmZl9zZF9yYXRpb2AgaXMgZXhwcmVzc2VkIGJ5ICQkXHRhdV97a2p9ID0gXGZyYWN7XGhhdHtcbXV9X2sgLSBcaGF0e1xtdX1fan17XGhhdHtcc2lnbWF9X2t9JCQgd2hlcmUgJGokIGlzIHRoZSBydW5uZXItdXAgcHJlZGljdGlvbi4gJFx0YXVfe2prfSQgZ2l2ZXMgdXMgYSByYXRpbyBvZiB0d28gZGlmZmVyZW50IG1lYXN1cmVzIG9mIHVuY2VydGFpbnR5LgoKIyBFeHBsb3JhdG9yeSBhbmFseXNpcwoKVGhpcyBzZWN0aW9uIHdpbGwgYmUgZGl2aWRlZCBpbnRvIHRvIHBhcnRzLiBGaXJzdCwgd2Ugd2lsbCBleGFtaW5lIHRoZSByZXN1bHRpbmcgZGF0YSBmcm9tIGBtb2RlbDU1YCwgd2hpY2ggd2lsbCBiZSByZWdhcmRlZCBhcyBvdXIgYmFzZWxpbmUgbW9kZWwuIE5leHQsIHdlIHdpbGwgYW5hbHlzZSB0aGUgZGF0YSBmcm9tIG1vZGVscyBvYnRhaW5lZCBieSB2YXJ5aW5nIGtlcm5lbCBzaXplcyBhbmQgZHJvcG91dCByYXRlcy4gIAoKIyMgVW5jZXJ0YWludHkgYW5hbHlzaXMgb2YgYmFzZWxpbmUgbW9kZWwgey50YWJzZXR9CgojIyMgRW50aXJlIGRhdGEgc2V0CgpgYGB7cn0KIyBJbXBvcnRpbmcgZGF0YQpkYXRhIDwtIGFzLnRpYmJsZShyZWFkLmNzdigifi9Eb2N1bWVudHMvTWFzdGVyb3BwZ2F2ZS9EYXRhL1Jlc3VsdGF0ZXIvcmVzdWx0c19zZ2RfZGZfMjAxODA0MDQuY3N2IikpCmRmIDwtIHNlbGVjdChkYXRhLCAtWCkKCiMgU3VtbWFyaXppbmcgZW50aXJlIGRhdGEgc2V0CnN1bW1hcnkoZGYpCmBgYApGb3IgdGhlIGVudGlyZSBzZXQgb2YgY2xhc3NpZmljYXRpb25zLCB3ZSBoYXZlIHRoZSBmb2xsb3dpbmcgbm90YWJsZSBxdWFudGl0aWVzOgoKKiBUaGUgbWVhbiAqKnVuY2VydGFpbnR5Kiogb2YgdGhlIHByZWRpY3RlZCBjbGFzcyBpcyBgciBtZWFuKGRmJHVuY2VydGFpbnR5KWAgYW5kIHRoZSBtZWRpYW4gaXMgYHIgbWVkaWFuKGRmJHVuY2VydGFpbnR5KWAuIFRoZSBtaW5pbXVtIHZhbHVlIGlzIGByIG1pbihkZiR1bmNlcnRhaW50eSlgLCB0aGUgbWF4aW11bSBpcyBgciBtYXgoZGYkdW5jZXJ0YWludHkpYC4gVGhlIGludGVycXVhcnRpbGUgcmFuZ2UgKElRUikgaXMgYHIgSVFSKGRmJHVuY2VydGFpbnR5KWAuCgoqIFRoZSBtZWFuICoqc29mdG1heCBvdXRwdXQgb2YgdGhlIHByZWRpY3RlZCBjbGFzcyoqIGlzIGByIG1lYW4oZGYkcHJvYjEpYCBhbmQgdGhlIG1lZGlhbiBpcyBgciBtZWRpYW4oZGYkcHJvYjEpYC4gVGhlIG1pbmltdW0gdmFsdWUgaXMgYHIgbWluKGRmJHByb2IxKWAsIHRoZSBtYXhpbXVtIGlzIGByIG1heChkZiRwcm9iMSlgLiBUaGUgSVFSIGlzIGByIElRUihkZiRwcm9iMSlgLgoKKiBUaGUgbWVhbiAqKnNvZnRtYXggb3V0cHV0IG9mIHRoZSBydW5uZXItdXAqKiBpcyBgciBtZWFuKGRmJHByb2IyKWAgYW5kIHRoZSBtZWRpYW4gaXMgYHIgbWVkaWFuKGRmJHByb2IyKWAuIFRoZSBtaW5pbXVtIHZhbHVlIGlzIGByIG1pbihkZiRwcm9iMilgLCB0aGUgbWF4aW11bSBpcyBgciBtYXgoZGYkcHJvYjIpYC4gVGhlIElRUiBpcyBgciBJUVIoZGYkcHJvYjIpYC4KCiogVGhlIG1lYW4gKipkaWZmZXJlbmNlKiogYmV0d2VlbiB0aGUgc29mdG1heCBvdXB1dHMgb2YgdGhlIHByZWRpY3Rpb24gYW5kIHJ1bm5lci11cCBpcyBgciBtZWFuKGRmJGRpZmYpYCBhbmQgdGhlIG1lZGlhbiBpcyBgciBtZWRpYW4oZGYkZGlmZilgLiBUaGUgbWluaW11bSB2YWx1ZSBpcyBgciBtaW4oZGYkZGlmZilgLCB0aGUgbWF4aW11bSBpcyBgciBtYXgoZGYkZGlmZilgLiBUaGUgSVFSIGlzIGByIElRUihkZiRkaWZmKWAuCgoqIFRoZSBtZWFuICoqZGlmZmVyZW5jZSB0byB1bmNlcnRhaW50eSByYXRpbyoqLCBvciAkXHRhdV97amt9JCwgaXMgYHIgbWVhbihkZiRkaWZmX3NkX3JhdGlvKWAgYW5kIHRoZSBtZWRpYW4gaXMgYHIgbWVkaWFuKGRmJGRpZmZfc2RfcmF0aW8pYC4gVGhlIG1pbmltdW0gdmFsdWUgaXMgYHIgbWluKGRmJGRpZmZfc2RfcmF0aW8pYCwgdGhlIG1heGltdW0gaXMgYHIgbWF4KGRmJGRpZmZfc2RfcmF0aW8pYC4gVGhlIElRUiBpcyBgciBJUVIoZGYkZGlmZl9zZF9yYXRpbylgLgoKIyMjIFN1bW1hcnkgc3RhdGlzdGljcwpgYGB7cn0KIyBBZ2dyZWdhdGluZyBzdW1tYXJ5IHN0YXRpc3RpY3MgYnkgY29ycmVjdC9pbmNvcnJlY3QKYWdnX2RmIDwtIGRmICU+JSAKICBncm91cF9ieShjb3JyZWN0KSAlPiUgCiAgc3VtbWFyaXNlKG49bigpLAogICAgICAgICAgICBtZWFuX3Byb2IxPW1lYW4ocHJvYjEpLAogICAgICAgICAgICBtZWFuX3Byb2IyPW1lYW4ocHJvYjIpLAogICAgICAgICAgbWVhbl91bmNlcnRhaW50eT1tZWFuKHVuY2VydGFpbnR5KSwKICAgICAgICAgIG1lZGlhbl91bmNlcnRhaW50eT1tZWRpYW4odW5jZXJ0YWludHkpLAogICAgICAgICAgc2RfdW5jZXJ0YWludHk9c2QodW5jZXJ0YWludHkpLCAKICAgICAgICAgIG1lYW5fZGlmZj1tZWFuKGRpZmYpLAogICAgICAgICAgbWVkaWFuX2RpZmY9bWVkaWFuKGRpZmYpLAogICAgICAgICAgc2RfZGlmZj1zZChkaWZmKSwKICAgICAgICAgIG1lYW5fcmF0aW89bWVhbihkaWZmX3NkX3JhdGlvKSwKICAgICAgICAgIG1lZGlhbl9yYXRpbz1tZWRpYW4oZGlmZl9zZF9yYXRpbyksCiAgICAgICAgICBzZF9yYXRpbz1zZChkaWZmX3NkX3JhdGlvKSkKYWdnX2RmCmBgYAoqIFRoZSBtb2RlbCBoYXMgKippbmNvcnJlY3RseSBjbGFzc2lmaWVkKiogYHIgYWdnX2RmJG5bMV1gIGltYWdlcy4KCiogVGhlIG1vZGVsIGhhcyAqKmNvcnJlY3RseSBjbGFzc2lmaWVkKiogYHIgYWdnX2RmJG5bMl1gIGltYWdlcy4KCiMjIyBJbmNvcnJlY3QgY2xhc3NpZmljYXRpb25zCmBgYHtyfQojIFN1bW1hcml6aW5nIGluY29ycmVjdCBwcmVkaWN0aW9ucwppbmNvcnJlY3RfZGYgPC0gZGYgJT4lIAogIGZpbHRlcihjb3JyZWN0PT0wKQpzdW1tYXJ5KGluY29ycmVjdF9kZikKYGBgCkZvciB0aGUgZW50aXJlIHNldCBvZiAqKmluY29ycmVjdCBjbGFzc2lmaWNhdGlvbnMqKiwgd2UgaGF2ZSB0aGUgZm9sbG93aW5nIG5vdGFibGUgcXVhbnRpdGllczoKCiogVGhlIG1lYW4gKip1bmNlcnRhaW50eSoqIG9mIHRoZSBwcmVkaWN0ZWQgY2xhc3MgaXMgYHIgbWVhbihpbmNvcnJlY3RfZGYkdW5jZXJ0YWludHkpYCBhbmQgdGhlIG1lZGlhbiBpcyBgciBtZWRpYW4oaW5jb3JyZWN0X2RmJHVuY2VydGFpbnR5KWAuIFRoZSBtaW5pbXVtIHZhbHVlIGlzIGByIG1pbihpbmNvcnJlY3RfZGYkdW5jZXJ0YWludHkpYCwgdGhlIG1heGltdW0gaXMgYHIgbWF4KGluY29ycmVjdF9kZiR1bmNlcnRhaW50eSlgLiBUaGUgaW50ZXJxdWFydGlsZSByYW5nZSAoSVFSKSBpcyBgciBJUVIoaW5jb3JyZWN0X2RmJHVuY2VydGFpbnR5KWAuCgoqIFRoZSBtZWFuICoqc29mdG1heCBvdXRwdXQgb2YgdGhlIHByZWRpY3RlZCBjbGFzcyoqIGlzIGByIG1lYW4oaW5jb3JyZWN0X2RmJHByb2IxKWAgYW5kIHRoZSBtZWRpYW4gaXMgYHIgbWVkaWFuKGluY29ycmVjdF9kZiRwcm9iMSlgLiBUaGUgbWluaW11bSB2YWx1ZSBpcyBgciBtaW4oaW5jb3JyZWN0X2RmJHByb2IxKWAsIHRoZSBtYXhpbXVtIGlzIGByIG1heChpbmNvcnJlY3RfZGYkcHJvYjEpYC4gVGhlIElRUiBpcyBgciBJUVIoaW5jb3JyZWN0X2RmJHByb2IxKWAuCgoqIFRoZSBtZWFuICoqc29mdG1heCBvdXRwdXQgb2YgdGhlIHJ1bm5lci11cCoqIGlzIGByIG1lYW4oaW5jb3JyZWN0X2RmJHByb2IyKWAgYW5kIHRoZSBtZWRpYW4gaXMgYHIgbWVkaWFuKGluY29ycmVjdF9kZiRwcm9iMilgLiBUaGUgbWluaW11bSB2YWx1ZSBpcyBgciBtaW4oaW5jb3JyZWN0X2RmJHByb2IyKWAsIHRoZSBtYXhpbXVtIGlzIGByIG1heChpbmNvcnJlY3RfZGYkcHJvYjIpYC4gVGhlIElRUiBpcyBgciBJUVIoaW5jb3JyZWN0X2RmJHByb2IyKWAuCgoqIFRoZSBtZWFuICoqZGlmZmVyZW5jZSoqIGJldHdlZW4gdGhlIHNvZnRtYXggb3VwdXRzIG9mIHRoZSBwcmVkaWN0aW9uIGFuZCBydW5uZXItdXAgaXMgYHIgbWVhbihpbmNvcnJlY3RfZGYkZGlmZilgIGFuZCB0aGUgbWVkaWFuIGlzIGByIG1lZGlhbihpbmNvcnJlY3RfZGYkZGlmZilgLiBUaGUgbWluaW11bSB2YWx1ZSBpcyBgciBtaW4oaW5jb3JyZWN0X2RmJGRpZmYpYCwgdGhlIG1heGltdW0gaXMgYHIgbWF4KGluY29ycmVjdF9kZiRkaWZmKWAuIFRoZSBJUVIgaXMgYHIgSVFSKGluY29ycmVjdF9kZiRkaWZmKWAuCgoqIFRoZSBtZWFuICoqZGlmZmVyZW5jZSB0byB1bmNlcnRhaW50eSByYXRpbyoqLCBvciAkXHRhdV97amt9JCwgaXMgYHIgbWVhbihpbmNvcnJlY3RfZGYkZGlmZl9zZF9yYXRpbylgIGFuZCB0aGUgbWVkaWFuIGlzIGByIG1lZGlhbihpbmNvcnJlY3RfZGYkZGlmZl9zZF9yYXRpbylgLiBUaGUgbWluaW11bSB2YWx1ZSBpcyBgciBtaW4oaW5jb3JyZWN0X2RmJGRpZmZfc2RfcmF0aW8pYCwgdGhlIG1heGltdW0gaXMgYHIgbWF4KGluY29ycmVjdF9kZiRkaWZmX3NkX3JhdGlvKWAuIFRoZSBJUVIgaXMgYHIgSVFSKGluY29ycmVjdF9kZiRkaWZmX3NkX3JhdGlvKWAuCgojIyMgQ29ycmVjdCBjbGFzc2lmaWNhdGlvbnMKCmBgYHtyIH0KIyBTdW1tYXJpemluZyBjb3JyZWN0IHByZWRpY3Rpb25zCmNvcnJlY3RfZGYgPC0gZGYgJT4lIAogIGZpbHRlcihjb3JyZWN0PT0xKQpzdW1tYXJ5KGNvcnJlY3RfZGYpCmBgYApGb3IgdGhlIGVudGlyZSBzZXQgb2YgKipjb3JyZWN0IGNsYXNzaWZpY2F0aW9ucyoqLCB3ZSBoYXZlIHRoZSBmb2xsb3dpbmcgbm90YWJsZSBxdWFudGl0aWVzOgoKKiBUaGUgbWVhbiAqKnVuY2VydGFpbnR5Kiogb2YgdGhlIHByZWRpY3RlZCBjbGFzcyBpcyBgciBtZWFuKGNvcnJlY3RfZGYkdW5jZXJ0YWludHkpYCBhbmQgdGhlIG1lZGlhbiBpcyBgciBtZWRpYW4oY29ycmVjdF9kZiR1bmNlcnRhaW50eSlgLiBUaGUgbWluaW11bSB2YWx1ZSBpcyBgciBtaW4oY29ycmVjdF9kZiR1bmNlcnRhaW50eSlgLCB0aGUgbWF4aW11bSBpcyBgciBtYXgoY29ycmVjdF9kZiR1bmNlcnRhaW50eSlgLiBUaGUgaW50ZXJxdWFydGlsZSByYW5nZSAoSVFSKSBpcyBgciBJUVIoY29ycmVjdF9kZiR1bmNlcnRhaW50eSlgLgoKKiBUaGUgbWVhbiAqKnNvZnRtYXggb3V0cHV0IG9mIHRoZSBwcmVkaWN0ZWQgY2xhc3MqKiBpcyBgciBtZWFuKGNvcnJlY3RfZGYkcHJvYjEpYCBhbmQgdGhlIG1lZGlhbiBpcyBgciBtZWRpYW4oY29ycmVjdF9kZiRwcm9iMSlgLiBUaGUgbWluaW11bSB2YWx1ZSBpcyBgciBtaW4oY29ycmVjdF9kZiRwcm9iMSlgLCB0aGUgbWF4aW11bSBpcyBgciBtYXgoY29ycmVjdF9kZiRwcm9iMSlgLiBUaGUgSVFSIGlzIGByIElRUihjb3JyZWN0X2RmJHByb2IxKWAuCgoqIFRoZSBtZWFuICoqc29mdG1heCBvdXRwdXQgb2YgdGhlIHJ1bm5lci11cCoqIGlzIGByIG1lYW4oY29ycmVjdF9kZiRwcm9iMilgIGFuZCB0aGUgbWVkaWFuIGlzIGByIG1lZGlhbihjb3JyZWN0X2RmJHByb2IyKWAuIFRoZSBtaW5pbXVtIHZhbHVlIGlzIGByIG1pbihjb3JyZWN0X2RmJHByb2IyKWAsIHRoZSBtYXhpbXVtIGlzIGByIG1heChjb3JyZWN0X2RmJHByb2IyKWAuIFRoZSBJUVIgaXMgYHIgSVFSKGNvcnJlY3RfZGYkcHJvYjIpYC4KCiogVGhlIG1lYW4gKipkaWZmZXJlbmNlKiogYmV0d2VlbiB0aGUgc29mdG1heCBvdXB1dHMgb2YgdGhlIHByZWRpY3Rpb24gYW5kIHJ1bm5lci11cCBpcyBgciBtZWFuKGNvcnJlY3RfZGYkZGlmZilgIGFuZCB0aGUgbWVkaWFuIGlzIGByIG1lZGlhbihjb3JyZWN0X2RmJGRpZmYpYC4gVGhlIG1pbmltdW0gdmFsdWUgaXMgYHIgbWluKGNvcnJlY3RfZGYkZGlmZilgLCB0aGUgbWF4aW11bSBpcyBgciBtYXgoY29ycmVjdF9kZiRkaWZmKWAuIFRoZSBJUVIgaXMgYHIgSVFSKGNvcnJlY3RfZGYkZGlmZilgLgoKKiBUaGUgbWVhbiAqKmRpZmZlcmVuY2UgdG8gdW5jZXJ0YWludHkgcmF0aW8qKiwgb3IgJFx0YXVfe2prfSQsIGlzIGByIG1lYW4oY29ycmVjdF9kZiRkaWZmX3NkX3JhdGlvKWAgYW5kIHRoZSBtZWRpYW4gaXMgYHIgbWVkaWFuKGNvcnJlY3RfZGYkZGlmZl9zZF9yYXRpbylgLiBUaGUgbWluaW11bSB2YWx1ZSBpcyBgciBtaW4oY29ycmVjdF9kZiRkaWZmX3NkX3JhdGlvKWAsIHRoZSBtYXhpbXVtIGlzIGByIG1heChjb3JyZWN0X2RmJGRpZmZfc2RfcmF0aW8pYC4gVGhlIElRUiBpcyBgciBJUVIoY29ycmVjdF9kZiRkaWZmX3NkX3JhdGlvKWAuCgojIyBEaXN0cmlidXRpb24gb2YgdW5jZXJ0YWludHkgey50YWJzZXR9CgpJbiB0aGUgZm9sbG93aW5nIHdlIHdpbGwgdmlzdWFsaXplIHRoZSByZWxhdGlvbnNoaXBzIGJldHdlZW4gb3VyIHZhcmlhYmVscy4gV2Ugc3RhcnQgYnkgZXhhbWluaW5nIHRoZSBlbXBpcmljYWwgZGlzdHJpYnV0aW9uIG9mIHRoZSB1bmNlcnRhaW50eSBlc3RpbWF0ZXMgJFxoYXR7XHNpZ21hfV9rJC4KCiMjIyBGdWxsIGRpc3RyaWJ1dGlvbgoKVGhlIGRpc3RyaWJ1dGlvbiBhcHBlYXJzIHRvIGJlIGJpbW9kYWwsIHdpdGggcGVha3MgY2xvc2UgdG8gMCBhbmQgMC4yOgoKYGBge3IgZmlnLmhlaWdodD00LCBmaWcud2lkdGg9Nn0KIyBEaXN0cmlidXRpb24gb2YgZXN0aW1hdGVkIHVuY2VydGFpbnR5CnAxIDwtIGRmICU+JSAKICBnZ3Bsb3QoYWVzKHg9dW5jZXJ0YWludHkpKSArCiAgZ2VvbV9oaXN0b2dyYW0oY29sPSJncmV5IiwgYmlucyA9IDUwLCBhbHBoYT0uNSkgKwogIGdndGl0bGUoIkRpc3RyaWJ1dGlvbiBvZiBlc3RpbWF0ZWQgdW5jZXJ0YWludHkiKQpwMQpgYGAKCiMjIyBVbmNlcnRhaW50eSBieSBwcmVkaWN0aW9uCgpCeSBncm91cGluZyB0aGUgdW5jZXJ0YWludHkgZXN0aW1hdGVzIGJ5IGBjb3JyZWN0YCAoaS5lLiBpZiB0aGUgbGFiZWwgd2FzIGNvcnJlY3RseSBwcmVkaWN0ZWQgb3Igbm90KSwgd2UgY2FuIGZpbmQgb3V0IGhvdyB0aGUgcHJlZGljdGlvbnMgY29udHJpYnV0ZSB0byB0aGUgdW5jZXJ0YWludHkgZGlzdHJpYnV0aW9uLgoKVGhlIGJsdWUgbGluZSBjb3JyZXNwb25kcyB0byB0aGUgY29ycmVjdCBwcmVkaWN0aW9ucywgdGhlIHJlZCBsaW5lIGNvcnJlc3BvbmRzIHRvIGluY29ycmVjdCBwcmVkaWN0aW9ucy4gV2Ugc2VlIHRoYXQgdGhlIGluY29ycmVjdCBwcmVkaWN0aW9ucyBhcmUgY2VudGVyZWQgYXJvdW5kIGEgaGlnaGVyIGFzc29jaWF0ZWQgdW5jZXJ0YWludHksIHdoZXJlYXMgZmFyIG1vcmUgb2YgdGhlIGNvcnJlY3RseSBwcmVkaWN0ZWQgY2xhc3NlcyBhcmUgY29uY2VudHJhdGVkIGFyb3VuZCBhIGxvdyB1bmNlcnRhaW50eSB2YWx1ZS4gVGhlIGluY29ycmVjdCBjbGFzc2lmaWNhdGlvbnMgZ3JlYXRseSBjb250cmlidXRlIHRvIHRoZSBiaW1vZGFsaXR5LCBidXQgaXQgaXMgYWxzbyBwcmVzZW50IGluIHRoZSBkaXN0cmlidXRpb24gb2YgdW5jZXJ0YWludHkgZm9yIHRoZSBjb3JyZWN0IGNsYXNzaWZpY2F0aW9ucy4gCgpgYGB7cn0KI0Rpc3RyaWJ1dGlvbiBvZiBlc3RpbWF0ZWQgdW5jZXJ0YWludHkgYnkgcHJlZGljdGlvbgpwMiA8LSBkZiAlPiUgCiAgZ2dwbG90KGFlcyh4PXVuY2VydGFpbnR5LCBjb2w9ZmFjdG9yKGNvcnJlY3QpKSkgKwogIGdlb21fZnJlcXBvbHkoYWxwaGE9LjcpICsKICBnZ3RpdGxlKCJEaXN0cmlidXRpb24gb2YgZXN0aW1hdGVkIHVuY2VydGFpbnR5IGJ5IGNsYXNzaWZpY2F0aW9uIikgKwogIHNjYWxlX2NvbG9yX2Rpc2NyZXRlKG5hbWU9IlByZWRpY3Rpb24iLAogICAgICAgICAgICAgICAgICAgICAgICAgYnJlYWtzPWMoIjAiLCAiMSIpLAogICAgICAgICAgICAgICAgICAgICAgICAgbGFiZWxzPWMoIjA6IEluY29ycmVjdCIsICIxOiBDb3JyZWN0IikpCnAyCmBgYAoKIyMjIEtlcm5lbCBkZW5zaXR5IGVzdGltYXRlcwoKVGhlIGZvbGxvd2luZyBrZXJuZWwgZGVuc2l0eSBlc3RpbWF0ZSBwbG90ICh1c2luZyBhIEdhdXNzaWFuIGtlcm5lbCkgZ2l2ZXMgdXMgYW4gaWRlYSBvZiBob3cgdGhlIGRpc3RyaWJ1dGlvbnMgY29tcGFyZSB0byBlYWNob3RoZXI6CgpgYGB7cn0KIyBLREUgYnkgY29ycmVjdCBwcmVkaWN0aW9uCnAzIDwtIGRmICU+JSAKICBnZ3Bsb3QoYWVzKHg9dW5jZXJ0YWludHkpKSArCiAgZ2VvbV9kZW5zaXR5KGRhdGE9aW5jb3JyZWN0X2RmLCBmaWxsPSJyZWQiLCBhbHBoYT1JKC4yKSkgKwogIGdlb21fZGVuc2l0eShkYXRhPWNvcnJlY3RfZGYsIGZpbGw9InR1cnF1b2lzZSIsIGFscGhhPUkoLjIpKSArCiAgZ2d0aXRsZSgiS2VybmVsIGRlbnNpdHkgZXN0aW1hdGVzIG9mIHVuY2VydGFpbnR5IGRpc3RyaWJ1dGlvbiIpCnAzCmBgYAoKIyMjIEJveHBsb3RzCgpUaGUgYm94cGxvdCBnaXZlcyB1cyB5ZXQgYW5vdGhlciB3YXkgdG8gdmlzdWFsaXplIHRoZSBkaWZmZXJlbmNlIGJldHdlZW4gdW5jZXJ0YWludHkgZGlzdHJpYnV0aW9ucyBieSBwcmVkaWN0aXZlIGFiaWxpdHkuIEl0IGlzIGludGVyZXN0aW5nIHRvIG5vdGUgdGhlIGFtb3VudCBvZiBvdXRsaWVycyBmb3IgdGhlIGluY29ycmVjdCBwcmVkaWN0aW9ucywgaW5kaWNhdGluZyBzb20gbmVnYXRpdmUgc2tldy4gSXQgaXMgd3JvbmcsIGJ1dCB1bmNlcnRhaW50eSBpcyBsb3cuCgpgYGB7cn0KIyBCb3hwbG90IG9mIHVuY2VydGFpbnRpZXMgZm9yIGNvcnJlY3QgdnMuIGluY29ycmVjdApwNCA8LSBkZiAlPiUgCiAgZ2dwbG90KGFlcyh4PWZhY3Rvcihjb3JyZWN0KSwgeT11bmNlcnRhaW50eSkpICsKICBnZW9tX2JveHBsb3QoYWVzKGZpbGw9ZmFjdG9yKGNvcnJlY3QpKSwgYWxwaGE9LjcpICsKICBsYWJzKHg9ImNvcnJlY3QiKSArCiAgZ2d0aXRsZSgiQm94cGxvdCBvZiB1bmNlcnRhaW50eSBkaXN0cmlidXRpb24gYnkgY29ycmVjdC9pbmNvcnJlY3QiKSArCiAgc2NhbGVfZmlsbF9kaXNjcmV0ZShuYW1lPSJQcmVkaWN0aW9uIiwKICAgICAgICAgICAgICAgICAgICAgICAgIGJyZWFrcz1jKCIwIiwgIjEiKSwKICAgICAgICAgICAgICAgICAgICAgICAgIGxhYmVscz1jKCIwOiBJbmNvcnJlY3QiLCAiMTogQ29ycmVjdCIpKQpwNApgYGAKCiMjIFJlbGF0aW9uc2hpcCB0byBvdGhlciB2YXJpYWJsZXMgey50YWJzZXR9CgojIyMgU29mdG1heCBvZiBwcmVkaWN0ZWQgY2xhc3MKCldlIG1heSBhbHNvIGJlIGludGVyZXN0ZWQgaW4gdGhlIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIGB1bmNlcnRhaW50eWAgYW5kIG90aGVyIHZhcmlhYmxlcy4gRmlyc3QsIHdlIHBsb3QgYHVuY2VydGFpbnR5YCBhZ2FpbnN0IGBwcm9iMWAgKHRoZSBwcmVkaWN0aW9uJ3Mgc29mdG1heCBvdXRwdXQpLiBUaGUgc29mdG1heCBvdXRwdXRzIGluIHRoZSBhYm92ZSBwbG90IGFyZSBjb2xvdXIgZ3JhZGVkLiBPdXRwdXRzIGNsb3NlIHRvIDEgYXJlIHJlZCwgb3V0cHV0cyBjbG9zZSB0byAwIGFyZSBibHVlLiBUaGUgcGxvdCBzaG93cyBhIGNsZWFyIHBhcmFib2xpYyBzaGFwZToKCmBgYHtyfQojIFBsb3R0aW5nIHNvZnRtYXggb3V0cHV0IG9mIHByZWRpY3Rpb24gYWdhaW5zdCBlc3RpbWF0ZWQgdW5jZXJ0YWludHkKcDUgPC0gZGYgJT4lIAogIGdncGxvdChhZXMoeD1wcm9iMSwgeT11bmNlcnRhaW50eSkpICsKICBnZW9tX3BvaW50KGFscGhhPS4yKSArCiAgZ2d0aXRsZSgiVW5jZXJ0YWludHkgdnMuIHNvZnRtYXggb3V0cHV0IG9mIHByZWRpY3Rpb24gZm9yIGFsbCBvYnNlcnZhdGlvbnMiKSArCiAgbGFicyh4PSJzb2Z0bWF4IG91dHB1dCBvZiBwcmVkaWN0ZWQgY2xhc3MiKSArCiAgc2NhbGVfY29sb3JfZGlzdGlsbGVyKG5hbWU9IlNvZnRtYXggb3V0cHV0IiwKICAgICAgICAgICAgICAgICAgICAgICAgcGFsZXR0ZSA9ICJTcGVjdHJhbCIpCnA1CmBgYAoKIyMjIEJ5IGNsYXNzaWZpY2F0aW9uClJlZCBwb2ludHMgaW5kaWNhdGUgaW5jb3JyZWN0IGNsYXNzaWZpY2F0aW9ucywgYmx1ZSBwb2ludHMgaW5kaWNhdGUgY29ycmVjdCBjbGFzc2lmaWNhdGlvbnMuCmBgYHtyfQpwNiA8LSBkZiAlPiUgCiAgbXV0YXRlKGNvcnJlY3Q9YXMubG9naWNhbChjb3JyZWN0KSkgJT4lIAogIGdncGxvdChhZXMoeD1wcm9iMSwgeT11bmNlcnRhaW50eSkpICsKICBnZW9tX3BvaW50KGFlcyhmaWxsPWNvcnJlY3QsIGNvbD1jb3JyZWN0KSwgc2hhcGU9MjEsIGFscGhhPS41KSArCiAgZ2d0aXRsZSgiVW5jZXJ0YWludHkgdnMuIHNvZnRtYXggb3V0cHV0IG9mIHByZWRpY3Rpb24gZm9yIGFsbCBvYnNlcnZhdGlvbnMiKSArCiAgbGFicyh4PSJzb2Z0bWF4IG91dHB1dCBvZiBwcmVkaWN0ZWQgY2xhc3MiKQpwNgpgYGAKCgojIyMgUHJlZGljdGlvbnMgYW5kIHJ1bm5lcnMtdXAKCldlIGNhbiBvYnRhaW4gbW9yZSBpbmZvcm1hdGlvbiBieSBjb2xvdXJpbmcgdGhlIHBvaW50cyBieSB0aGUgdmFsdWUgb2YgdGhlIHJ1bm5lci11cCBwcmVkaWN0aW9ucy4gVGhpcyBwbG90IGlzIHBhcnRpY3VsYXJseSBpbnRlcmVzdGluZy4gVGhlIHNvZnRtYXggb3V0cHV0cyBvZiB0aGUgcnVubmVyLXVwIGNsYXNzZXMgJFxoYXR7XG11fV9qJCBpbiB0aGUgYWJvdmUgcGxvdCBhcmUgY29sb3VyIGdyYWRlZC4gJFxoYXR7XG11fV9qIFxhcHByb3ggMC41JCBhcmUgcmVkLCAkXGhhdHtcbXV9X2ogXGFwcHJveCAwJCBhcmUgYmx1ZS4gVGhlIHJvdW5kIHBvaW50IGlzIHRoZSBwYWlyIG1lYW4gdmFsdWVzIG9mIChgcHJvYjFgLCBgcHJvYjJgKSBmb3IgdGhlIGluY29ycmVjdCBwcmVkaWN0aW9ucy4gVGhlIHRyaWFuZ3VsYXIgcG9pbnQgaXMgdGhlIHBhaXIgbWVhbiB2YWx1ZXMgZm9yIHRoZSBjb3JyZWN0IHByZWRpY3Rpb25zLgoKV2Ugc2VlIGEgY2xlYXIgY29uY2VudHJhdGlvbiBvZiByZWQgcG9pbnRzIGluIHRoZSBhcmVhIHdoZXJlIHRoZSBwcm9iYWJpbGl0eSBvZiB0aGUgcHJlZGljdGllZCBjbGFzcyAkXGhhdHtcbXV9X2sgXGFwcHJveCAwLjUkLiBJZiB0aGUgc29mdG1heCBwcmVkaWN0aW9ucyBvZiBib3RoIHRoZSBwcmVkaWN0ZWQgY2xhc3MgYW5kIHRoZSBydW5uZXItdXAgYXJlIGNsb3NlIDAuNSwgdGhlbiB3ZSBoYXZlIGEgc2l0dWF0aW9uIGFuYWxvZ291cyB0byBtYXhpbXVtIGVudHJvcHkuIFRoaXMgcG9pbnRzIGNvaW5jaWRlIHdpdGggdGhlIGxhcmdlc3QgYXBwcm94aW1hdGVkIHVuY2VydGFpbnR5IHZhbHVlcy4gCgpgYGB7cn0KIyBQbG90dGluZyBzb2Z0bWF4IG91dHB1dCBvZiBwcmVkaWN0aW9uIGFnYWluc3QgZXN0aW1hdGVkIHVuY2VydGFpbnR5LCBjb2xvdXJlZCBieSBzb2Z0bWF4IG91dHB1dCBvZiBydW5uZXItdXAKcDcgPC0gZGYgJT4lIAogIGdncGxvdChhZXMoeD1wcm9iMSwgeT11bmNlcnRhaW50eSkpICsKICBnZW9tX3BvaW50KGFlcyhjb2w9cHJvYjIpLCBzaGFwZT0yMSwgYWxwaGE9LjcpICsKICBnZW9tX3BvaW50KGRhdGE9YWdnX2RmLCBhZXMoeD1tZWFuX3Byb2IxLCB5PW1lYW5fcHJvYjIsIHNoYXBlPWFzLmxvZ2ljYWwoY29ycmVjdCkpLCBzaXplPTIuNSkgKwogICNnZW9tX3BvaW50KGRhdGE9YWdnX2RmLCBhZXMoeT11bmNlcnRhaW50eSwgeD1tZWFuX3Byb2IxLCBzaGFwZT1hcy5sb2dpY2FsKGNvcnJlY3QpKSwgc2l6ZT0yKSArCiAgZ2d0aXRsZSgiVW5jZXJ0YWludHkgdnMuIHNvZnRtYXggb3V0cHV0IG9mIHByZWRpY3Rpb24gZm9yIGFsbCBvYnNlcnZhdGlvbnMiKSArCiAgbGFicyh4PSJzb2Z0bWF4IG91dHB1dCBvZiBwcmVkaWN0ZWQgY2xhc3MiLCBzaGFwZT0iY29ycmVjdCIpICsKICBzY2FsZV9jb2xvcl9kaXN0aWxsZXIobmFtZT0icnVubmVyIHVwIiwKICAgICAgICAgICAgICAgICAgICAgICAgcGFsZXR0ZSA9ICJTcGVjdHJhbCIpCnA3CmBgYAoKIyMjIFNvZnRtYXggb2YgcHJlZGljdGlvbiBieSBjbGFzc2lmaWNhdGlvbgoKSW4gdGhlIGZvbGxvd2luZyBwbG90IHdlIHNwbGl0IHRoZSBvYnNlcnZhdGlvbnMgYnkgaW5jb3JyZWN0L2NvcnJlY3QgcHJlZGljdGlvbnMsIGFuZCBwbG90IHRoZSB2YWx1ZXMgb2YgdW5jZXJ0YWludHkgYWdhaW5zdCB0aGUgc29mdG1heCBvdXRwdXQgb2YgdGhlIHByZWRpY3RlZCBjbGFzczoKCmBgYHtyIGZpZy53aWR0aD0xMCwgZmlnLmhlaWdodD01fQojIFBsb3R0aW5nIHVuY2VydGFpbnR5IHZzLiBzb2Z0bWF4IG91dHB1dCBvZiBwcmVkaWN0aW9uCnA4YSA8LSBkZiAlPiUgCiAgZ2dwbG90KGFlcyh4PXByb2IxLCB5PXVuY2VydGFpbnR5KSkgKwogIGdlb21fcG9pbnQoYWVzKGNvbD1wcm9iMiksIHNoYXBlPTIxLCBhbHBoYT0uNykgKwogIGdndGl0bGUoIlVuY2VydGFpbnR5IHZzLiBzb2Z0bWF4IG91dHB1dCBvZiBwcmVkaWN0aW9uIGJ5IGluY29ycmVjdC9jb3JyZWN0IikgKwogIGxhYnMoeD0ic29mdG1heCBvdXRwdXQgb2YgcHJlZGljdGlvbiIpICsKICBmYWNldF9ncmlkKC5+ZmFjdG9yKGNvcnJlY3QpKSArCiAgc2NhbGVfY29sb3JfZGlzdGlsbGVyKG5hbWU9IlJ1bm5lciB1cCIsCiAgICAgICAgICAgICAgICAgICAgICAgIHBhbGV0dGUgPSAiU3BlY3RyYWwiKQpwOGEKYGBgCiMjIyBTb2Z0bWF4IG9mIHByZWRpY3Rpb24gYnkgY2xhc3NpZmljYXRpb24gd2l0aCBjb250b3VycwoKVGhlIGJsYWNrIGNvbnRvdXIgbGluZXMgaW5kaWNhdGUgd2hlcmUgbW9zdCBvZiB0aGUgcG9pbnRzIGFyZSBjb25jZW50cmF0ZWQuIFRoZSBwbG90IG9uIHRoZSBsZWZ0IGlzIGZvciBpbmNvcnJlY3QgcHJlZGljdGlvbnMuIFRoZSByaWdodCBoYW5kIHBsb3QgcmVwcmVzZW50cyB0aGUgY29ycmVjdCBwcmVkaWN0aW9ucy4gRm9yIHRoZSBjb3JyZWN0IHByZWRpY3Rpb25zLCBpdCBzZWVtcyBhcyBpZiBmYXIgbW9yZSBvZiB0aGUgcG9pbnRzIGFyZSBjb25jZW50cmF0ZWQgYXJvdW5kIGhpZ2ggcHJlZGljdGVkIG91dHB1dC9sb3cgcnVubmVyLXVwIG91dHB1dC9sb3cgdW5jZXJ0YWludHkuIFRoaXMgaXMgbm90IHN1cnByaXNpbmcsIGNvbnNpZGVyaW5nIDc1JSBvZiB0aGUgY29ycmVjdCBwcmVkaWN0aW9ucyBoYXZlIGEgc29mdG1heCB2YWx1ZSBvZiBhcHByb3hpbWF0ZWx5IDAuNyBvciBhYm92ZS4gRm9yIHRoZSBpbmNvcnJlY3QgY2xhc3NpZmljYXRpb25zLCBtb3N0IG9mIHRoZSBwb2ludHMgYXJlIGNvbmNlbnRyYXRlZCBhcm91bmQgdGhlIGFyZWEgb2YgbWF4aW11bSBlbnRyb3B5LiBUaGlzIGluZGljYXRlcyB0aGF0IHRoZSBhcHByb3hpbWF0ZWQgdW5jZXJ0YWludHkgZXN0aW1hdGVzIGluZGVlZCBjb250YWluIHZhbHVhYmxlIGluZm9ybWF0aW9uIGluIHRoZSBpbmNvcnJlY3QgY2FzZXMuCmBgYHtyIGZpZy53aWR0aD0xMCwgZmlnLmhlaWdodD01fQojIFBsb3R0aW5nIHVuY2VydGFpbnR5IHZzLiBzb2Z0bWF4IG91dHB1dCBvZiBwcmVkaWN0aW9uCnA4IDwtIGRmICU+JSAKICBnZ3Bsb3QoYWVzKHg9cHJvYjEsIHk9dW5jZXJ0YWludHkpKSArCiAgZ2VvbV9wb2ludChhZXMoY29sPXByb2IyKSwgc2hhcGU9MjEsIGFscGhhPS43KSArCiAgZ2VvbV9kZW5zaXR5XzJkKGNvbD0iYmxhY2siLCBhbHBoYT0uMykgKwogIGdndGl0bGUoIlVuY2VydGFpbnR5IHZzLiBzb2Z0bWF4IG91dHB1dCBvZiBwcmVkaWN0aW9uIGJ5IGluY29ycmVjdC9jb3JyZWN0IikgKwogIGxhYnMoeD0ic29mdG1heCBvdXRwdXQgb2YgcHJlZGljdGlvbiIpICsKICBmYWNldF9ncmlkKC5+ZmFjdG9yKGNvcnJlY3QpKSArCiAgc2NhbGVfY29sb3JfZGlzdGlsbGVyKG5hbWU9IlJ1bm5lciB1cCIsCiAgICAgICAgICAgICAgICAgICAgICAgIHBhbGV0dGUgPSAiU3BlY3RyYWwiKQpwOApgYGAKCiMjIyBSdW5uZXItdXAgcHJlZGljdGlvbnMKClRoZSBmb2xsb3dpbmcgcGxvdCBzaG93cyB0aGUgcmVsYXRpb25zaGlwIGJldHdlZW4gdW5jZXJ0YWludHkgZXN0aW1hdGVzIGFuZCB0aGUgc29mdG1heCBvdXRwdXQgb2YgdGhlIHJ1bm5lci11cCBwcmVkaWN0aW9uLiBVbnN1cnByaXNpbmdseSwgbW9kZWwgdW5jZXJ0YWludHkgaW5jcmVhc2VzIGFzIHRoZSBzb2Z0bWF4IG91dHB1dCBvZiB0aGUgcnVubmVyLXVwIGluY3JlYXNlcy4gV2UgaGF2ZSBwbG90dGVkIGEgTE9FU1MgZXN0aW1hdGUgb2YgdGhlIG1lYW4gdW5jZXJ0YWludHkgYXMgYSBmdW5jdGlvbiBvZiB0aGUgcnVubmVyLXVwIG91dHB1dCB0byBtYWtlIHRoaXMgY2xlYXJlcjoKCmBgYHtyfQojIFBsb3R0aW5nIHNvZnRtYXggb3V0cHV0IG9mIHJ1bm5lci11cCBhZ2FpbnN0IGVzdGltYXRlZCB1bmNlcnRhaW50eQpwMTAgPC0gZGYgJT4lIAogIGdncGxvdChhZXMoeD1wcm9iMiwgeT11bmNlcnRhaW50eSkpICsKICBnZW9tX3BvaW50KGFscGhhPS4yKSArCiAgZ2VvbV9zbW9vdGgobWV0aG9kPSJsb2VzcyIpICsKICBsYWJzKHg9InNvZnRtYXggb3V0cHV0IG9mIHJ1bm5lci11cCIpICsKICBnZ3RpdGxlKCJVbmNlcnRhaW50eSB2cy4gc29mdG1heCBvdXRwdXQgb2YgcnVubmVyLXVwIikKcDEwCmBgYAoKIyMgSW1hZ2VzIGFzc29jaWF0ZWQgd2l0aCBoaWdoIHVuY2VydGFpbnR5IHsudGFic2V0fQoKVGhlIGZvbGxvd2luZyBwbG90cyB3ZXJlIGdlbmVyYXRlZCB1c2luZyBQeXRob24gKGNvZGUgYXZhaWxhYmxlIG9uIFtHaXRIdWJdKGh0dHBzOi8vZ2l0aHViLmNvbS9zbXUwOTUvc211MDk1LmdpdGh1Yi5pby9ibG9iL21hc3Rlci9kYXQyNTkuaXB5bmIpKS4gT24gdGhlIGxlZnQgaGFuZCBzaWRlIHdlIHNlZSB0aGUgdW5ub3JtYWxpemVkIGltYWdlIHdpdGggdGhlIGNvcnJlc3BvbmRpbmcgZ3JvdW5kIHRydXRoIGxhYmVsLiBUaGUgcGxvdCBpbiB0aGUgbWlkZGxlIHNob3dzIHRoZSBzb2Z0bWF4IG91dHB1dCBvZiB0aGUgcHJlZGljdGVkIGNsYXNzIGZvciBlYWNoIG9mIHRoZSAkVD0xMDAkIHN0b2NoYXN0aWMgZm9yd2FyZCBwYXNzZXMuICRcbXVfayQgaXMgZ2l2ZW4gYnkgdGhlIHNvbGlkIHJlZCBsaW5lLCAkXG11X2okIGlzIGdpdmVuIGJ5IHRoZSBkYXNoZWQgYnJvd24gbGluZS4gVGhlIHBsb3QgdGl0bGUgc2hvd3MgYm90aCB0aGUgcHJlZGljdGVkIGNsYXNzIGFuZCB0aGUgcnVubmVyLXVwIGNsYXNzLiBUbyB0aGUgcmlnaHQgaXMgYSBrZXJuZWwgZGVuc2l0eSBlc3RpbWF0ZSAodXNpbmcgYSBHYXVzc2lhbiBrZXJuZWwpIG9mIHRoZSAkVD0xMDAkIHNvZnRtYXggb3V0cHV0cyBmb3IgdGhlIHByZWRpY3RlZCBjbGFzcy4gVGhlIHBsb3RzIHNob3cgdGhlIHRvcCA1IG1vc3QgdW5jZXJ0YWluIGNsYXNzaWZpY2F0aW9ucyBpbiB0aGUgZW50aXJlIGRhdGEgc2V0LgoKKipOT1RFOioqIEFkZCBuZXcgaW1hZ2VzLgoKIyMjIEluY29ycmVjdAoKIVtdKEZpZ3VyZXMvNDQ0Ml90Zi5qcGVnKQohW10oRmlndXJlcy8zMjg3X3RmLmpwZWcpCiFbXShGaWd1cmVzLzg0NDlfdGYuanBlZykKIVtdKEZpZ3VyZXMvNTgzOF90Zi5qcGVnKQohW10oRmlndXJlcy8zNTY5X3RmLmpwZWcpCgojIyMgQ29ycmVjdAohW10oRmlndXJlcy8xMjIzX3RmLmpwZWcpCiFbXShGaWd1cmVzLzYwMTBfdGYuanBlZykKIVtdKEZpZ3VyZXMvNDQ2OF90Zi5qcGVnKQohW10oRmlndXJlcy85ODE2X3RmLmpwZWcpCiFbXShGaWd1cmVzLzc0ODZfdGYuanBlZykKCiMjIFVuY2VydGFpbnR5LXByZWRpY3Rpb24gY29ycmVsYXRpb24KCkFzIG1lbnRpb25lZCBpbiBzZWN0aW9uIDIuMiwgdGhlIGNvbm5lY3Rpb24gZXN0YWJsaXNoZWQgYmV0d2VlbiBkcm9wb3V0IG5ldXJhbCBuZXR3b3JrcyBhbmQgR1BzIGFyZSBsb3N0IHdoZW4gYXBwbGllZCB0byBjb252b2x1dGlvbmFsIG5ldXJhbCBuZXR3b3Jrcy4gUGVyZm9ybWluZyBhIGxvZ2lzdGljIHJlZ3Jlc3Npb24gZ2l2ZXMgdXMgYSBzaW1wbGUgd2F5IG9mIHRlc3RpbmcgaWYgdGhlIGFwcHJveGltYXRlZCB1bmNlcnRhaW50eSBpcyBhIHNpZ25pZmljYW50IHByZWRpY3RvciBvZiB0aGUgbW9kZWwncyBhYmlsaXR5IHRvIHByZWRpY3QgY29ycmVjdGx5LgoKVGhlIGNvZWZmaWNpZW50IGFzc29jaWF0ZWQgd2l0aCBgdW5jZXJ0YWludHlgIGlzIGhpZ2hseSBzaWduaWZpY2FudCwgaW5kaWNhdGluZyB0aGF0IHRoZSBlc3RpbWF0ZXMgZ2F0aGVyZWQgZnJvbSBwZXJmb3JtaW5nIE1DIGRyb3BvdXQgYXJlIGluZGVlZCBhIHVzZWZ1bCBxdWFudGlmaWNhdGlvbiBvZiBwcmVkaWN0aXZlIHVuY2VydGFpbnR5LiBHaXZlbiBhIHVuaXQgaW5jcmVhc2UgaW4gdW5jZXJ0YWludHksIHdlIGV4cGVjdCB0aGUgcHJvYmFiaWxpdHkgb2YgcHJlZGljdGluZyBjb3JyZWN0bHkgdG8gZGVjcmVhc2UuCgpgYGB7cn0KIyBGaXR0aW5nIGxvZ2lzdGljIHJlZ3Jlc3Npb24gbW9kZWwgdG8gY2hlY2sgc2lnbmlmaWNhbmNlIG9mIHVuY2VydGFpbnR5Cm1vZGVsX3NkIDwtIGdsbShmYWN0b3IoY29ycmVjdCl+dW5jZXJ0YWludHksIGRhdGE9ZGYsIGZhbWlseSA9IGJpbm9taWFsKGxpbms9ImxvZ2l0IikpCnN1bW1hcnkobW9kZWxfc2QpCmBgYAoKIyMgUmVmZXJyYWwgY3JpdGVyaWEKClRoZSBxdWVzdGlvbiBpcyB0aGVuOiBIb3cgZG8gd2UgZGV0ZXJtaW5lIGEgcmVhc29uYWJsZSB1bmNlcnRhaW50eSB2YWx1ZSBmb3IgcmVmZXJyYWwgdG8gYSBodW1hbiBleHBlcnQ/IEFzIHdlIHNhdyBpbiBzZWN0aW9uIDMuMS4yLCB0aGUgbWVhbiB1bmNlcnRhaW50eSB2YWx1ZXMgb2YgdGhlIGluY29ycmVjdCBwcmVkaWN0aW9ucyBoaWdoZXIgdGhhbiBmb3IgdGhlIGNvcnJlY3QgcHJlZGljdGlvbnMgKGByIGFnZ19kZiRtZWFuX3VuY2VydGFpbnR5WzFdYCB2cy4gYHIgYWdnX2RmJG1lYW5fdW5jZXJ0YWludHlbMl1gLCByZXNwZWN0aXZlbHkpLgoKTmFpdmVseSwgd2UgY291bGQgc2V0IHRoZSB0aHJlc2hvbGQgZm9yIHJlZmVycmFsIHRvIHRoZSBtZWFuIHVuY2VydGFpbnR5IG9mIHRoZSBpbmNvcnJlY3RseSBjbGFzc2lmaWVkIGltYWdlczoKCmBgYHtyfQojIENvdW50aW5nIG51bWJlciBvZiBjb3JyZWN0L2luY29ycmVjdCBieSB1bmNlcnRhaW50eSA+PSAuMjA3IApyZWZlcnJhbF9tZWFuIDwtIGRmICU+JSAKICBmaWx0ZXIodW5jZXJ0YWludHk+PS4yMDcpICU+JSAKICBjb3VudChjb3JyZWN0KQpyZWZlcnJhbF9tZWFuCmBgYAoKVGhlIHByb2JsZW0gaGVyZSBpcyBhcHBhcmVudDogTWFueSBvZiB0aGUgY29ycmVjdGx5IGNsYXNzaWZpZWQgaW1hZ2VzIGFyZSBhc3NvY2lhdGVkIHdpdGggYSByZWxhdGl2ZWx5IGhpZ2ggbGV2ZWwgb2YgdW5jZXJ0YWludHkgKGluIG91ciBjYXNlLCB0aGlzIGlzIGR1ZSB0byB0aGUgYmltb2RhbGl0eSBvZiB0aGUgdW5jZXJ0YWludHkgZGlzdHJpYnV0aW9uIGluIHNlY3Rpb24gMy4yLjEpLiBUaGUgcHJvcG9ydGlvbiBvZiBpbmNvcnJlY3RseSBjbGFzc2lmaWVkIGltYWdlcyBpbiB0aGUgcmVmZXJyZWQgc2FtcGxlIGlzIGByIHJlZmVycmFsX21lYW4kblsxXS9zdW0ocmVmZXJyYWxfbWVhbiRuKWAuIAoKVGhlIHJlc3VsdCBvZiBvdXIgbG9naXN0aWMgcmVncmVzc2lvbiBzdWdnZXN0cyB0aGF0IHVuY2VydGFpbnR5IGlzIGEgc2lnbmlmaWNhbnQgcHJlZGljdG9yIG9mIGEgbW9kZWwncyBhYmlsaXR5IHRvIHByZWRpY3QgY29ycmVjdGx5LiBIb3dldmVyLCBhbHRob3VnaCB1bmNlcnRhaW50eSBzZWVtcyB0byBjb250YWluIHZhbHVlYWJsZSBpbmZvcm1hdGlvbiwgdGhlIHJlbGF0aXZlIHVuY2VydGFpbnRpZXMgb2YgdGhlIGNvcnJlY3QvaW5jb3JyZWN0IG9ic2VydmF0aW9ucyAoaW4gdGhpcyBjYXNlKSBtYXkgYmUgdG9vIHNtYWxsIHRvIGRpZmZlcmVudGlhdGUgd2hpY2ggaW1hZ2VzIHNob3VsZCBiZSByZWZlcnJlZCB0byBhbiBleHBlcnQuIFdlIG1heSBuZWVkIHRvIGFtcGxpZnkgdGhlIHF1YW50aWZpY2F0aW9uIG9mIHVuY2VydGFpbnR5IGluIHNvbWUgd2F5LgoKIyMjIFVzZWZ1bG5lc3Mgb2YgcnVubmVyLXVwIHByZWRpY3Rpb25zCgpGb3IgdGhlIG1vc3QgdW5jZXJ0YWluIGluY29ycmVjdGx5IGNsYXNzaWZpZWQgaW1hZ2VzIGluIHNlY3Rpb24gMy40LjEgdGhlIHJ1bm5lci11cCBzdWdnZXN0aW9ucyBhcmUgaW4gZmFjdCB0aGUgZ3JvdW5kIHRydXRoIGxhYmVscyA4MCUgb2YgdGhlIHRpbWUuIFRoaXMgbWF5IGJlIGR1ZSB0byBjaGFuY2UsIGJ1dCBzdGlsbCBiZWdzIHRoZSBxdWVzdGlvbjogSXMgdGhlcmUgYW55IGluZm9ybWF0aW9uIHRvIGJlIG9idGFpbmVkIGZyb20gdGhlIHJ1bm5lci11cCBwcmVkaWN0aW9ucz8gV2hhdCB3b3VsZCBoYXBwZW4gdG8gb3VyIG92ZXJhbGwgYWNjdXJhY3kgaWYgd2UgdXNlZCB0aGUgcnVubmVyLXVwIHByZWRpY3Rpb25zIGZvciBhbGwgaW5jb3JyZWN0IGNsYXNzaWZpY2F0aW9ucz8KCmBgYHtyfQojIENvdW50aW5nIGNsYXNzaWZpY2F0aW9uIGFjY3VyYWN5IGlmIHJ1bm5lci11cCBpcyBlcXVhbCB0byBncm91bmQgdHJ1dGgKY2xhc3MyX2RmIDwtIGRmICU+JQogIG11dGF0ZShjb3JyZWN0PXJlcGxhY2UoY29ycmVjdCwgY29ycmVjdD09MCAmIGNsYXNzMj09dHJ1dGgsIDEpKSAlPiUgCiAgY291bnQoY29ycmVjdCkKY2xhc3MyX2RmCmBgYAoKQWNjdXJhY3kgd291bGQgcmlzZSB0byBgciBjbGFzczJfZGYkblsyXS8oc3VtKGNsYXNzMl9kZiRuKSlgLiBUaGlzIGluZGljYXRlcyB0aGF0IHRoZXJlIG1heSBiZSBzb21lIHZhbHVhYmxlIGluZm9ybWF0aW9uIHRvIGJlIGdhdGhlcmVkIGZyb20gdGhlIHJ1bm5lci11cCBwcmVkaWN0aW9ucy4KCiMjIyBBIHJldmlzZWQgcXVhbnRpZmljYXRpb24gb2YgdW5jZXJ0YWludHkKCk9uZSBwb3NzaWJsZSBhcHByb2FjaCBpcyB0byB1c2UgdGhlIHZhbHVlIG9mICRcdGF1X3tqa30kIGludHJvZHVjZWQgaW4gc2VjdGlvbiAyLjUuIFJlY2FsbCB0aGF0ICRcdGF1X3tqa30kIGluY29ycG9yYXRlcyBpbmZvcm1hdGlvbiBhYm91dCB0aGUgcnVubmVyLXVwIHByZWRpY3Rpb24gaW50byBvdXIgcXVhbnRpZmljYXRpb24gb2YgdW5jZXJ0YWludHkuIENvbnNpZGVyIHRoZSBmb2xsb3dpbmcgY2FzZXM6CgoqICRcdGF1X3tran0gXGFwcHJveCAxJCBtZWFucyB0aGF0IGBkaWZmYCBhbmQgYHVuY2VydGFpbnR5YCBhcmUgcmVsYXRpdmVseSBzaW1pbGFyLiBUaGlzIGhhcHBlbnMgaWYgYSkgdGhlIG1vZGVscyBoYXZlIGZhaWxlZCB0byByZWFjaCBhIGNvbnNlbnN1cyAoYGRpZmZgIGlzIHNtYWxsKSBidXQgbW9kZWwgdW5jZXJ0YWludHkgaXMgbG93LCBvciBiKSB0aGUgbW9kZWxzIGhhdmUgcmVhY2hlZCBhIGNvbnNlbnN1cyAoYGRpZmZgIGlzIGxhcmdlKSBidXQgbW9kZWwgdW5jZXJ0YWludHkgaXMgaGlnaC4gTGV0J3MgY2FsbCB0aGVzZSAqKnJlZmVycmFsIHByZWRpY3Rpb25zKiouCgoqICRcdGF1X3tran0gXHRvIDAkIG1lYW5zIHRoYXQgYHVuY2VydGFpbnR5YCBpcyBtdWNoIGxhcmdlciB0aGFuIGBkaWZmYC4gVGhlc2Ugc2hvdWxkIHJlcHJlc2VudCAqKnVuY2VydGFpbiBwcmVkaWN0aW9ucyoqLgoKKiAkXHRhdV97a2p9IFx0byBcaW5mdHkkIG1lYW5zIHRoYXQgYHVuY2VydGFpbnR5YCBpcyBtdWNoIHNtYWxsZXIgdGhhbiBgZGlmZmAuIFRoZXNlIHNob3VsZCByZXByZXNlbnQgKipub24tcmVmZXJyYWwgcHJlZGljdGlvbnMqKi4KCkFzIHdlIHNhdyBpbiB0YWJsZSAzLjEuMiwgdGhlIG1lYW4gJFx0YXVfe2prfSQgZm9yIHRoZSBpbmNvcnJlY3RseSBjbGFzc2lmaWVkIGltYWdlcyBpcyBgciBhZ2dfZGYkbWVhbl9yYXRpb1sxXWAgYW5kIGByIGFnZ19kZiRtZWFuX3JhdGlvWzJdYCBmb3IgdGhlIGNvcnJlY3RseSBjbGFzc2lmaWVkIGltYWdlcy4gVGhlIHJlc3BlY3RpdmUgbWVkaWFucyBhcmUgYHIgYWdnX2RmJG1lZGlhbl9yYXRpb1sxXWAgYW5kIGByIGFnZ19kZiRtZWRpYW5fcmF0aW9bMl1gLgoKQXMgYSBmaXJzdCBzdGVwLCBzZXR0aW5nIHRoZSByZWZlcnJhbCB0aHJlc2hvbGQgdG8gJFx0YXVfe2prfSBcbGVxIDEuODMkICh0aGUgbWVhbiB1bmNlcnRhaW50eSBvZiB0aGUgaW5jb3JyZWN0IGNsYXNzaWZpY2F0aW9ucykgZ2l2ZXM6CgpgYGB7cn0KIyBDb3VudGluZyBudW1iZXIgb2YgY29ycmVjdC9pbmNvcnJlY3QgYnkgdGF1IDw9IDIuOApyZWZlcnJhbF9tZWFudGF1IDwtIGRmICU+JSAKICBmaWx0ZXIoZGlmZl9zZF9yYXRpbyA8PSAxLjgzKSAlPiUgCiAgY291bnQoY29ycmVjdCkKcmVmZXJyYWxfbWVhbnRhdQpgYGAKCldlIHJ1biBpbnRvIHRoZSBzYW1lIHByb2JsZW0gYXMgd2hlbiB3ZSB1c2VkIHRoZSBtZWFuIHVuY2VydGFpbnR5OiBNYW55IG9mIHRoZSBjb3JyZWN0bHkgcHJlZGljdGVkIGltYWdlcyB3b3VsZCBiZSByZWZlcnJlZC4gSXQgaXMgaW50ZXJlc3RpbmcgdG8gbm90ZSwgaG93ZXZlciwgdGhhdCB3ZSBnZXQgbW9yZSByZWZlcnJhbHMgb2YgaW5jb3JyZWN0bHkgY2xhc3NpZmllZCBpbWFnZXMuIEluIHRoaXMgY2FzZSwgdGhlIHByb3BvcnRpb24gb2YgaW5jb3JyZWN0bHkgY2xhc3NpZmllZCBpbWFnZXMgaW4gdGhlIHJlZmVycmVkIHNhbXBsZSBpcyBgciByZWZlcnJhbF9tZWFudGF1JG5bMV0vc3VtKHJlZmVycmFsX21lYW50YXUkbilgLiAKClVwb24gaW5zcGVjdGlvbiwgdGhlIGJveHBsb3Qgb2YgdGhlIGxvZy10cmFuc2Zvcm1lZCAkXHRhdV97amt9JCBjbGVhcmx5IHNob3dzIHRoZSBwcmVzZW5jZSBvZiBleHRyZW1lIHZhbHVlcyAob3V0bGllcnMgbWFya2VkIGluIHJlZCkgaW4gYm90aCB0YWlscy4gRm9yIGluY29ycmVjdCBwcmVkaWN0aW9ucywgdGhlIGRpc3RyaWJ1dGlvbiBzZWVtcyB0byBiZSBuZWdhdGl2ZWx5IHNrZXdlZC4gVGhlIGRpc3RyaWJ1dGlvbiBvZiB0aGUgY29ycmVjdCBwcmVkaWN0aW9ucyBhcHBlYXIgdG8gYmUgcG9zaXRpdmVseSBza2V3ZWQuIAoKYGBge3J9CnB0IDwtIGRmICU+JSAKICBnZ3Bsb3QoYWVzKHg9ZmFjdG9yKGNvcnJlY3QpLCB5PWxvZyhkaWZmX3NkX3JhdGlvKSkpICsKICBnZW9tX2JveHBsb3QoYWVzKGZpbGw9ZmFjdG9yKGNvcnJlY3QpKSwgb3V0bGllci5jb2xvdXIgPSAicmVkIiwgb3V0bGllci5hbHBoYSA9IC4xNSkgKwogIHhsYWIoImNvcnJlY3QiKSArCiAgeWxhYigibG9nLXRyYW5zZm9ybWVkIHRhdSIpICArCiAgZ2d0aXRsZSgiUHJlc2VuY2Ugb2YgZXh0cmVtZSB1bmNlcnRhaW50eSB2YWx1ZXMiKSArCiAgbGFicyhmaWxsPSJjb3JyZWN0IikKcHQKYGBgCgoKUGVyaGFwcyBhIG1vcmUgcm9idXN0IG1lYXN1cmUgb2YgdGhlIGNlbnRyYWwgdGVuZGVuY3kgb2YgJFx0YXVfe2prfSQgaXMgdGhlIG1lZGlhbi4gU2V0dGluZyB0aGUgcmVmZXJyYWwgcmF0ZSBvZiAkXHRhdV97amt9IFxsZXEgMS4yJCBnaXZlcyB0aGUgZm9sbG93aW5nOgoKYGBge3J9CiMgQ291bnRpbmcgbnVtYmVyIG9mIGNvcnJlY3QvaW5jb3JyZWN0IGJ5IHRhdSA8PSAxCnJlZmVycmFsX21lZHRhdSA8LSBkZiAlPiUgCiAgZmlsdGVyKGRpZmZfc2RfcmF0aW8gPD0gLjk2NSkgJT4lIAogIGNvdW50KGNvcnJlY3QpCnJlZmVycmFsX21lZHRhdQpgYGAKClRoaXMgaXMgYSBjbGVhciBpbXByb3ZlbWVudCBvdmVyIG91ciB0aGUgbWVhbiBhcHByb2FjaCwgaW4gdGVybXMgb2YgdGhlIGFtb3VudCBvZiByZWZlcnJlZCBpbWFnZXMuIGByIGFicyhyZWZlcnJhbF9tZWR0YXUkblsxXS1yZWZlcnJhbF9tZWFudGF1JG5bMV0pYCBmZXdlciBpbmNvcnJlY3QgaW1hZ2VzIGFyZSByZWZlcnJlZCBjb21wYXJlZCB0byBgciBhYnMocmVmZXJyYWxfbWVkdGF1JG5bMl0tcmVmZXJyYWxfbWVhbnRhdSRuWzJdKWAgZmV3ZXIgY29ycmVjdCBpbWFnZXMuIEluIHRoaXMgY2FzZSwgdGhlIHByb3BvcnRpb24gb2YgaW5jb3JyZWN0bHkgY2xhc3NpZmllZCBpbWFnZXMgaW4gdGhlIHJlZmVycmVkIHNhbXBsZSBpcyBgciByZWZlcnJhbF9tZWR0YXUkblsxXS9zdW0ocmVmZXJyYWxfbWVkdGF1JG4pYC4KCldoZW4gY29tcGFyZWQgdG8gdXNpbmcgdGhlIG1lYW4gb2YgJFxoYXR7XHNpZ21hfV9rJCBhcyBhIGN1dC1vZmYgZm9yIHJlZmVycmFsLCB0aGUgbWVkaWFuIG9mICRcdGF1X3tqa30kIHJlc3VsdHMgaW4gYWxtb3N0IDIwJSBtb3JlIHJlZmVycmFscyBvZiBpbmNvcnJlY3QgaW1hZ2UgcmVsYXRpdmUgdG8gdGhlIHNhbXBsZSBzaXplLiBUaGlzIGNvdWxkIGluZGljYXRlIHRoYXQgJFx0YXVfe2prfSQgaXMgYSBtb3JlIHNlbnNpdGl2ZSB1bmNlcnRhaW50eSBxdWFudGlmaWNhdGlvbiBmb3IgcHJhY3RpY2FsIGFwcGxpY2F0aW9ucy4KCiMjIENvbXBhcmlzb24gb2YgdGF1IGFuZCB1bmNlcnRhaW50eXsudGFic2V0fQoKV2UgY2FuIGV4YW1pbmUgdGhpcyBmdXJ0aGVyIGJ5IHBsb3R0aW5nIHRoZSBudW1iZXIgb2YgY29ycmVjdC9pbmNvcnJlY3QgcmVmZXJyYWxzIGFnYWluc3QgZGlmZmVyZW50IHZhbHVlcyBmb3IgJFx0YXVfe2prfSQgYW5kICRcc2lnbWFfayQuCgojIyMgRGF0YSBwcmVwYXJhdGlvbgpGaXJzdCB3ZSBuZWVkIHRvIHByZXBhcmUgdGhlIGRhdGEgZm9yIHBsb3R0aW5nLiBJbiB0aGUgZm9sbG93aW5nIGNodW5rIHdlIGNvdW50IHRoZSBudW1iZXIgb2YgcmVmZXJyYWxzIG9mIGNvcnJlY3RseS9pbmNvcnJlY3RseSBjbGFzc2lmaWVkIGltYWdlcyBieSBpbmNyZWFzaW5nIHRoZSB2YWx1ZXMgb2YgJFx0YXVfe2prfSQgYW5kICRcc2lnbWFfayQ6CgoqIEZvciAkXHRhdV97amt9JCwgd2Ugc2V0IHRoZSBpbml0aWFsIHZhbHVlICRcdGF1XzEgPSAwLjA1JCBhbmQgaW5jcmVtZW50IGJ5IDAuMDEgdW50aWwgd2UgcmVhY2ggYSBtYXhpbXVtIHZhbHVlIG9mIDEwLiBUaGlzIGN1dC1vZmYgdmFsdWUgd2FzIGNob3NlbiBiZWNhdXNlIGl0IGdpdmVzIHVzIHJvdWdobHkgdGhlIHNhbWUgcHJvcG9ydGlvbiBvZiBjb3JyZWN0IHZzLiBpbmNvcnJlY3QgY2xhc3NpZmljYXRpb25zIGFzIG91ciB0aHJlc2hvbGQgZm9yICRcc2lnbWFfayQuIEZvciBldmVyeSAkXHRhdV97amt9IFxsZXEgXHRhdV9pJCB3ZSBjb3VudCB0aGUgbnVtYmVyIGNvcnJlY3QgYW5kIGluY29ycmVjdCBjbGFzc2lmaWNhdGlvbnMgb2YgdGhlIHJlZmVycmVkIGltYWdlcy4gRmluYWxseSwgd2UgY2FsY3VsYXRlIHRoZSByZWxhdGl2ZSBwcm9wb3J0aW9ucyBvZiBjb3JyZWN0bHkvaW5jb3JyZWN0bHkgY2xhc3NpZmllZCBpbWFnZXMgcGVyIHZhbHVlLgoKKiBGb3IgJFxzaWdtYV9rJCwgd2Ugc2V0IHRoZSBpbml0aWFsIHZhbHVlIHRvICRcc2lnbWFfMT0wLjAxJCBhbmQgaW5jcmVtZW50IGJ5IDAuMDEgdW50aWwgd2UgcmVhY2ggYSBtYXhpbXVtIHZhbHVlIG9mIC4zNS4gVGhpcyBjdXQtb2ZmIHZhbHVlIHdhcyBjaG9zZW4gYmVjYXVzZSBpdCBnaXZlcyB1cyByb3VnaGx5IHRoZSBzYW1lIHByb3BvcnRpb24gb2YgY29ycmVjdCB2cy4gaW5jb3JyZWN0IGNsYXNzaWZpY2F0aW9ucyBhcyBvdXIgdGhyZXNob2xkIGZvciAkXHRhdV97amt9JC4gRm9yIGV2ZXJ5ICRcc2lnbWFfe2t9IFxnZXEgXHNpZ21hX2kkIChpLmUuIGFsbCBpbWFnZXMgdGhhdCBoYXZlIGhpZ2hlciB1bmNlcnRhaW50eSB0aGFuIHRoaXMgdmFsdWUpIHdlIGNvdW50IHRoZSBudW1iZXIgY29ycmVjdCBhbmQgaW5jb3JyZWN0IGNsYXNzaWZpY2F0aW9ucyBvZiB0aGUgcmVmZXJyZWQgaW1hZ2VzLgoKYGBge3J9CiMgUHJlcGFyaW5nIGRhdGEKY291bnRfZGYgPC0gZGYgJT4lIAogIHJlbmFtZSh0YXU9ZGlmZl9zZF9yYXRpbykgJT4lIAogIHNlbGVjdChjb3JyZWN0LCB1bmNlcnRhaW50eSwgdGF1KQoKIyBHYXRoZXJpbmcgbnVtYmVyIG9mIHJlZmVycmFscyBieSB0YXUKcm9jX3RhdSA8LSBkYXRhLmZyYW1lKG5fZmFsc2U9bnVtZXJpYygpLCBuX3RydWU9bnVtZXJpYygpLCB0YXVfdmFsdWU9bnVtZXJpYygpKQppZHg9MQp0YXVfc2VxIDwtIHNlcSgwLjA1LCAxMCwgYnk9LjEpCgpmb3IodmFsdWUgaW4gdGF1X3NlcSl7CiAgZ2V0X3Jlc3VsdHMgPC0gY291bnRfZGYgJT4lCiAgICBmaWx0ZXIodGF1PD12YWx1ZSkgJT4lIAogICAgY291bnQoY29ycmVjdCkKICByb2NfdGF1W2lkeCxdIDwtIGMoZ2V0X3Jlc3VsdHMkbiwgdmFsdWUpCiAgaWR4IDwtIGlkeCArIDEKfQoKcGxvdF90YXUgPC0gcm9jX3RhdSAlPiUgCiAgZ2F0aGVyKHN0YXR1cywgbiwgLXRhdV92YWx1ZSkgJT4lIAogIGdyb3VwX2J5KHRhdV92YWx1ZSkgJT4lIAogIG11dGF0ZShwPW4vc3VtKG4pKSAlPiUgCiAgdW5ncm91cCgpCgojIyBHYXRoZXJpbmcgbnVtYmVyIG9mIHJlZmVycmFscyBwZXIgYnkgdW5jZXJ0YWludHkKcm9jX3VuYyA8LSBkYXRhLmZyYW1lKG5fZmFsc2U9bnVtZXJpYygpLCBuX3RydWU9bnVtZXJpYygpLCB1bmNlcnRhaW50eV92YWx1ZT1udW1lcmljKCkpCmlkeD0xCnVuY2VydGFpbnR5X3NlcSA8LSBzZXEoMC4wMSwgLjM1LCBieT0uMDEpCgpmb3IodmFsdWUgaW4gdW5jZXJ0YWludHlfc2VxKXsKICBnZXRfcmVzdWx0cyA8LSBjb3VudF9kZiAlPiUKICAgIGZpbHRlcih1bmNlcnRhaW50eT49dmFsdWUpICU+JSAKICAgIGNvdW50KGNvcnJlY3QpCiAgcm9jX3VuY1tpZHgsXSA8LSBjKGdldF9yZXN1bHRzJG4sIHZhbHVlKQogIGlkeCA8LSBpZHggKyAxCn0KCnBsb3RfdW5jIDwtIHJvY191bmMgJT4lIAogIGdhdGhlcihzdGF0dXMsIG4sIC11bmNlcnRhaW50eV92YWx1ZSkgJT4lIAogIGdyb3VwX2J5KHVuY2VydGFpbnR5X3ZhbHVlKSAlPiUgCiAgbXV0YXRlKHA9bi9zdW0obikpICU+JSAKICB1bmdyb3VwKCkKYGBgCgojIyMgUGxvdHRpbmcKVGhlIGZpcnN0IHJvdyBvZiB0aGUgcGxvdCBjb250YWlucyB0aGUgcmVmZXJyYWxzIGZvciB2YXJ5aW5nIHZhbHVlcyBvZiAkXHNpZ21hX2skLiBUaGUgc2Vjb25kIHJvdyBzaG93cyByZWZlcnJhbCBjb3VudHMgZm9yIHZhcnlpbmcgdmFsdWVzIG9mICRcdGF1X3tqa30kLiBUaGUgZmlyc3QgY29sdW1uIHNob3cgdGhlIGFic29sdXRlIGNvdW50cyAobm90ZSB0aGUgZGlmZmVyZW50IHgtYXhlcykgYW5kIHRoZSBzZWNvbmQgY29sdW1uIHNob3dzIHRoZSByZWxhdGl2ZSByZWZlcnJhbCByYXRlcy4gVGhlIGRhc2hlZCBob3Jpc29udGFsIGxpbmVzIHJlcHJlc2VudCB0aGUgbWVhbiBvZiAkXHNpZ21hX2skIGFuZCB0aGUgbWVkaWFuIG9mICRcdGF1X3tqa30kLCByZXNwZWN0aXZlbHksIGZvciB0aGUgaW5jb3JyZWN0bHkgY2xhc3NpZmllZCBpbWFnZXMuCgpgYGB7ciBmaWcuaGVpZ2h0PTYsIGZpZy53aWR0aD04fQojIEFic29sdXRlIGNvdW50cwpwMWEgPC0gZ2dwbG90KHBsb3RfdW5jLCBhZXMoeD11bmNlcnRhaW50eV92YWx1ZSwgeT1uLCBncm91cD1zdGF0dXMpKSArCiAgZ2VvbV9saW5lKGFlcyhjb2w9c3RhdHVzKSkgKwogIGdlb21fdmxpbmUoeGludGVyY2VwdD1hZ2dfZGYkbWVhbl91bmNlcnRhaW50eVsxXSwgbHdkPS4zLCBsdHk9ImRhc2hlZCIpICsKICBsYWJzKHg9InVuY2VydGFpbnR5IiwgeT0ibnVtYmVyIG9mIHJlZmVycmFscyIpICsKICBnZ3RpdGxlKCJBYnNvbHV0ZSByZWZlcnJhbCBjb3VudHMiKSArCiAgdGhlbWUobGVnZW5kLnBvc2l0aW9uID0gIm5vbmUiKQoKcDJhIDwtIGdncGxvdChwbG90X3RhdSwgYWVzKHg9dGF1X3ZhbHVlLCB5PW4sIGdyb3VwPXN0YXR1cykpICsKICBnZW9tX2xpbmUoYWVzKGNvbD1zdGF0dXMpKSArCiAgZ2VvbV92bGluZSh4aW50ZXJjZXB0PWFnZ19kZiRtZWRpYW5fcmF0aW9bMV0sIGx3ZD0uMywgbHR5PSJkYXNoZWQiKSArCiAgbGFicyh4PSJ0YXUiLCB5PSJudW1iZXIgb2YgcmVmZXJyYWxzIikgKwogIGdndGl0bGUoIkFic29sdXRlIHJlZmVycmFsIGNvdW50cyIpKwogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIikKCiMgUmVsYXRpdmUgY291bnRzCnAxYiA8LSBnZ3Bsb3QocGxvdF91bmMsIGFlcyh4PXVuY2VydGFpbnR5X3ZhbHVlLCB5PXAsIGdyb3VwPXN0YXR1cykpICsKICBnZW9tX2xpbmUoYWVzKGNvbD1zdGF0dXMpKSArCiAgbGFicyh4PSJ1bmNlcnRhaW50eSIsIHk9InByb3BvcnRpb24gb2YgcmVmZXJyYWxzIikgKwogIGdlb21fdmxpbmUoeGludGVyY2VwdD1hZ2dfZGYkbWVhbl91bmNlcnRhaW50eVsxXSwgbHdkPS4zLCBsdHk9ImRhc2hlZCIpICsKICBnZ3RpdGxlKCJSZWxhdGl2ZSByZWZlcnJhbCByYXRlcyIpKwogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIikKCnAyYiA8LSBnZ3Bsb3QocGxvdF90YXUsIGFlcyh4PXRhdV92YWx1ZSwgeT1wLCBncm91cD1zdGF0dXMpKSArCiAgZ2VvbV9saW5lKGFlcyhjb2w9c3RhdHVzKSkgKwogIGxhYnMoeD0idGF1IiwgeT0icHJvcG9ydGlvbiBvZiByZWZlcnJhbHMiKSArCiAgZ2VvbV92bGluZSh4aW50ZXJjZXB0PWFnZ19kZiRtZWRpYW5fcmF0aW9bMV0sIGx3ZD0uMywgbHR5PSJkYXNoZWQiKSArCiAgZ2d0aXRsZSgiUmVsYXRpdmUgcmVmZXJyYWwgcmF0ZXMiKSsKICB0aGVtZShsZWdlbmQucG9zaXRpb24gPSAibm9uZSIpCgpsYXlvdXQgPC0gbWF0cml4KGMoMSwyLDMsNCksIG5yb3cgPSAyLCBuY29sID0gMiwgYnlyb3c9VFJVRSkKbXVsdGlwbG90KHAxYSwgcDFiLCBwMmEsIHAyYiwgbGF5b3V0ID0gbGF5b3V0KQpgYGAKCiMjIyBUYXUgdnMuIHVuY2VydGFpbnR5CgpQbG90dGluZyB0aGUgYWJzb2x1dGUgcmVmZXJyYWwgY291bnRzIGZvciB2YXJ5aW5nIHZhbHVlcyBvZiAkXHRhdV97amt9JCAoYmx1ZSBsaW5lKSBhbmQgJFxzaWdtYV9rJCAocmVkIGxpbmUpIGluIHRoZSBzYW1lIHBsb3QgdGVsbHMgYXMgdGhhdCAkXHRhdV97amt9JCBnaXZlcyB1cyBtb3JlICJiYW5nIGZvciB0aGUgYnVjayIsIHNvIHRvIHNwZWFrLgoKYGBge3J9CnAzYSA8LSBnZ3Bsb3Qocm9jX3RhdSwgYWVzKHg9bl90cnVlLCB5PW5fZmFsc2UpKSArCiAgZ2VvbV9saW5lKGNvbD0iYmx1ZSIpICsKICBnZW9tX2xpbmUoZGF0YT1yb2NfdW5jLCBhZXMoeD1uX3RydWUsIHk9bl9mYWxzZSksIGNvbD0icmVkIikgKwogIHlsaW0oYygwLDIwMDApKSArCiAgeGxpbShjKDAsODAwMCkpCnAzYQpgYGAKCgojIERpZ2l0IGNsYXNzaWZpY2F0aW9uIG9uIE1OSVNUCgoqKkRyYWZ0OioqIENoZWNraW5nIHRvIHNlZSBpZiAkXHRhdV97amt9JCBpcyB1c2VmdWwgb24gYSBjb21wbGV0ZWx5IGRpZmZlcmVudCBkYXRhIHNldCAoTU5JU1QpLgoKYGBge3J9CiMgSW1wb3J0aW5nIE1OSVNUIGRhdGEKZGF0YV9tbmlzdCA8LSBhcy50aWJibGUocmVhZC5jc3YoIn4vRG9jdW1lbnRzL01hc3Rlcm9wcGdhdmUvRGF0YS9SZXN1bHRhdGVyL21uaXN0X2RmLmNzdiIpKQpkZl9tbmlzdCA8LSBzZWxlY3QoZGF0YV9tbmlzdCwgLVgpCgojIFN1bW1hcml6aW5nIGVudGlyZSBkYXRhIHNldApzdW1tYXJ5KGRmX21uaXN0KQpgYGAKCmBgYHtyfQojIEFnZ3JlZ2F0aW5nIHN1bW1hcnkgc3RhdGlzdGljcyBieSBjb3JyZWN0L2luY29ycmVjdAphZ2dfZGZfbW5pc3QgPC0gZGZfbW5pc3QgJT4lIAogIGdyb3VwX2J5KGNvcnJlY3QpICU+JSAKICBzdW1tYXJpc2Uobj1uKCksCiAgICAgICAgICAgIG1lYW5fcHJvYjE9bWVhbihwcm9iMSksCiAgICAgICAgICAgIG1lYW5fcHJvYjI9bWVhbihwcm9iMiksCiAgICAgICAgICBtZWFuX3VuY2VydGFpbnR5PW1lYW4odW5jZXJ0YWludHkpLAogICAgICAgICAgbWVkaWFuX3VuY2VydGFpbnR5PW1lZGlhbih1bmNlcnRhaW50eSksCiAgICAgICAgICBzZF91bmNlcnRhaW50eT1zZCh1bmNlcnRhaW50eSksIAogICAgICAgICAgbWVhbl9kaWZmPW1lYW4oZGlmZiksCiAgICAgICAgICBtZWRpYW5fZGlmZj1tZWRpYW4oZGlmZiksCiAgICAgICAgICBzZF9kaWZmPXNkKGRpZmYpLAogICAgICAgICAgbWVhbl90YXU9bWVhbihkaWZmX3NkX3JhdGlvKSwKICAgICAgICAgIG1lZGlhbl90YXU9bWVkaWFuKGRpZmZfc2RfcmF0aW8pLAogICAgICAgICAgc2RfcmF0aW89c2QoZGlmZl9zZF9yYXRpbykpCmFnZ19kZl9tbmlzdApgYGAKCmBgYHtyfQojIEZpbmRpbmcgY2FzZXMgd2hlcmUgcnVubmVyIHVwIGVxdWFscyB0cnVlIGNsYXNzCm1lZHRhdSA8LSBkZl9tbmlzdCAlPiUgCiAgZmlsdGVyKGNsYXNzMiA9PSB0cnV0aCkgJT4lIAogIHN1bW1hcmlzZShuID0gbigpLAogICAgbWVhbl91bmNlcnRhaW50eSA9IG1lYW4odW5jZXJ0YWludHkpLAogICAgICAgICAgICBtZWRpYW5fdW5jZXJ0YWludHkgPSBtZWRpYW4odW5jZXJ0YWludHkpLAogICAgICAgICAgICBtZWFuX3RhdSA9IG1lYW4oZGlmZl9zZF9yYXRpbyksCiAgICAgICAgICAgIG1lZGlhbl90YXUgPSBtZWRpYW4oZGlmZl9zZF9yYXRpbykpCm1lZHRhdQpgYGAKCmBgYHtyfQojIFNldHRpbmcgcmVmZXJyYWwgcmF0ZSB0byBtZWFuIHVuY2VydGFpbnR5IG9mIGluY29ycmVjdCBwcmVkaWN0aW9ucwpyZWZlcnJhbF9tbmlzdF9zaWdtYSA8LSBkZl9tbmlzdCAlPiUgCiAgZmlsdGVyKHVuY2VydGFpbnR5ID49IC4yMikgJT4lIAogIGNvdW50KGNvcnJlY3QpCnJlZmVycmFsX21uaXN0X3NpZ21hCmBgYApJbiB0aGlzIGNhc2UsIHRoZSBwcm9wb3J0aW9uIG9mIGluY29ycmVjdGx5IGNsYXNzaWZpZWQgaW1hZ2VzIGluIHRoZSByZWZlcnJlZCBzYW1wbGUgaXMgYHIgcmVmZXJyYWxfbW5pc3Rfc2lnbWEkblsxXS9zdW0ocmVmZXJyYWxfbW5pc3Rfc2lnbWEkbilgLgoKYGBge3J9CiMgU2V0dGluZyByZWZlcnJhbCByYXRlIHRvIG1lYW4gdW5jZXJ0YWludHkgb2YgaW5jb3JyZWN0IHByZWRpY3Rpb25zCnJlZmVycmFsX21uaXN0X3RhdSA8LSBkZl9tbmlzdCAlPiUgCiAgZmlsdGVyKGRpZmZfc2RfcmF0aW8gPD0gMS4zNikgJT4lIAogIGNvdW50KGNvcnJlY3QpCnJlZmVycmFsX21uaXN0X3RhdQpgYGAKCkluIHRoaXMgY2FzZSwgdGhlIHByb3BvcnRpb24gb2YgaW5jb3JyZWN0bHkgY2xhc3NpZmllZCBpbWFnZXMgaW4gdGhlIHJlZmVycmVkIHNhbXBsZSBpcyBgciByZWZlcnJhbF9tbmlzdF90YXUkblsxXS9zdW0ocmVmZXJyYWxfbW5pc3RfdGF1JG4pYC4gQWdhaW4sIHByZWxpbWluYXJ5IHJlc3VsdHMgc3VnZ2VzdCB0aGF0IHJ1bm5lci11cCBwcmVkaWN0aW9ucyBjYW4gYmUgbGV2ZXJhZ2VkIHRvIHByb2R1Y2Ugc2Vuc2libGUgY3V0LW9mZiB2YWx1ZXMgZm9yIHJlZmVycmFsLg==